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SPEND TAXPAYER MONEY LIKE IT WAS 
YOUR OWN 


In this overview of my seventh Annual Report to 
the Legislative Assembly, I want to highlight an 
issue that was apparent in a number of our value- 
for-money audits this year—specifically, public 
funds were often not being spent with enough due 
diligence and oversight to ensure that taxpayers 
were getting full value for their hard-earned tax 
dollars. For instance, we expressed concerns about 
ministries overpaying for goods or services, not 
ensuring that only those entitled to government 
benefits actually received them, not being diligent 
enough in collecting money owing to the govern- 
ment, and not ensuring that the level of user fees 
and premiums being charged is regularly reviewed 
and set at appropriate levels. 

It is not the absence or inadequacy of rules or 
guidelines that was the problem. Rather, I believe 
that there is a culture or mindset among some of 
those accountable for managing and delivering gov- 
ernment programs that does not always prioritize 
getting maximum value for the taxpayer’s dollar. 
So, what’s required to fix the problem? Essentially, 
the government and its employees—from the top 
leadership to those on the front lines delivering 
services to the public—must spend taxpayer money 
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like it was their own. Maximizing value for tax- 
payer dollars must be a priority at the top or it will 
certainly not be first and foremost in the minds of 
those responsible for actually delivering services to 
the public. 

In recent years, the economy has been strong 
and government revenues have often exceeded 
expectations. As a result, there has been sufficient 
cash available to ensure that all government 
programs and services could be delivered or even 
expanded, even if funds were not always being 
spent as well as they could have been. However, 
times have changed and revenue projections over 
the next few years are not rosy. Usually in such 
circumstances, individuals in a household instinct- 
ively adopt a belt-tightening mindset to ensure that 
the necessities can be provided for. Adopting this 
mindset throughout government will be more chal- 
lenging, but it will be no less essential if we are to 
ensure that the services that Ontarians have come 
to rely on can continue to be provided. 

It is atheme woven through a number of the 
value-for-money audits in this Annual Report—and 
in the Special Report on Ontario’s Electronic Health 
Records Initiative that we published in October— 
that the government needs to do a better job of 
monitoring how it spends taxpayers’ money and 
how it collects it. Some of the areas where we had 
concerns of this nature are summarized in the fol- 
lowing sections. 





Assistive Devices Program 
The Ministry of Health and Long-Term Care sets 
prices for assistive devices such as mobility devices, 
respiratory devices, and computer systems with the 
intent that the device suppliers obtain about a 33% 
rate of return. However, the prices that the Ministry 
has set for these assistive devices actually give 
vendors a rate of return that is often in the 100% 
range. As well, the Ministry was not being vigilant 
enough in following up on potential abuses in this 
program. 


Ontario Disability Support Program 
Income and employment support payments to 
individuals with eligible disabilities have risen by 
42% to more than $3 billion since our last audit 
in 2004. Staff of the Ministry of Community and 
Social Services were often not performing the 
required third-party verification of information 
provided by applicants to confirm their eligibility. 
Unrecovered overpayments increased substantially, 
to $663 million, up 37% since our last audit. Many 
of these overpayments could have been avoided 
if the Ministry had more effectively assessed the 
eligibility and the amounts to be paid to those indi- 
viduals and if it had followed up more diligently on 
tips received from the public. 


Ontario Works Program 
The Ministry of Community and Social Services 
paid out $1.7 billion in benefits for the Ontario 
Works Program last year. As with the Ontario Dis- 
ability Support Program (ODSP), the Ministry did 
not have adequate procedures in place to ensure 
that the required verification of third party informa- 
tion provided by applicants was being done. For the 
Ontario Works program, this is the responsibility of 
municipal service managers. As well, unrecovered 
overpayments have increased 45% since our last 
audit in 2002, from $414 million to $600 mil- 
lion, and there have been only minimal efforts to 
recover these overpayments. Given the challenge of 
recovering overpayments from both Ontario Works 
and ODSP recipients, it is all the more essential that 
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appropriate safeguards and controls be in place to 
prevent overpayments in the first place. 


OntarioBuys Program 
The purpose of the OntarioBuys Program is to help 
broader-public-sector (BPS) institutions such as 
hospitals, school boards, colleges, and universities 
achieve savings in their purchases of goods and ser- 
vices. The program funds “shared-service organiza- 
tions” that arrange for group purchasing of goods 
and services as well as projects aimed at improving 
organizations’ supply-chain practices. However, 
despite the program spending $148 million since 
its inception in 2004, the level of participation by 
BPS institutions, especially in the education sec- 
tor, was well below targeted levels. We also noted 
that the March 2009 Ontario Budget announced 
that OntarioBuys had helped BPS entities redirect 
$45 million in savings toward frontline services— 


but much of the $45 million in savings could not be 


substantiated, and almost all of those savings that 
could be substantiated were not actually spent on 
delivering frontline services to the public. 


Ontario Research Fund 
Since the Fund’s inception in 2004, about $300 mil- 
lion in research funding has been paid out, with 
total announced program commitments of more 
than $600 million. When the Fund was established, 
the Minister of Economic Development and Trade 
stated that one of the key objectives was to place 
a greater emphasis on commercialization and to 
“take good ideas out of our labs and ensure they get 
to the marketplace.” However, most of the research 
commitments to date have been theoretical in 
nature as opposed to research that has commercial 
potential. Given the Minister’s stated objective 
for this program—and in light of the impact of 
the recent recession on employment in Ontario— 
commercialization and the creation of new jobs 
should be more of a focus in order to fully meet the 
objectives of this program. 
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Government User Fees 
We identified issues with the way the government 
manages programs that bring in revenues through 
user fees, worth about $2.2 billion to the provincial 
treasury in the 2008/09 fiscal year. A user fee is 
generally charged to recover all or part of the cost 
of providing a specific good or service, such as 
a driver’s licence. In contrast, a tax is applied to 
produce revenues for general government purposes 
and for goods and services that the government 
deems to be a “public good,” such as health care. 

A 1998 Supreme Court of Canada decision 
concluded that a user fee could be considered 
unlawful—and therefore may have to be repaid to 
the user—either if it is determined by a court to be 
a tax with no law on the books to support it or if the 
amount charged is excessive when compared to the 
cost of the services provided. We noted that user 
fees worth more than $500 million a year might 
be invalid under the Supreme Court’s criteria. 
Although the government requires that user fees 
be regularly reviewed to keep them up-to-date, 
we found that there were generally no formal pro- 
cesses in place to ensure that this was being done 
effectively. 


Unfunded Liability of the Workplace Safety 

and Insurance Board 
One key objective of the Workplace Safety and 
Insurance Board (WSIB) is to run an insurance 
fund, funded by employers and investment returns 
on fund assets, that is used to provide income sup- 
port and medical assistance to workers injured on 
the job. As of December 31, 2008, the assets avail- 
able to pay the projected costs of injury claims were 
short by $11.4 billion. This is known as the fund’s 
“unfunded liability.” The challenge of trying to 
satisfy both workers—who want higher benefits— 
and employers—who want lower premiums—has 
contributed to this significant unfunded liability. 
The WSIB may need to find a new approach to deal- 
ing with this problem. Failure to do so could result 
in the WSIB ultimately being unable to meet its 


commitments to provide workers with the benefits 
to which they are entitled. 


THE SAFETY AND PROTECTION OF 
ONTARIANS 


Several of this year’s audits focused on programs 
that have an impact on the safety and protection of 
the public. While we noted some areas where good 
initiatives were being undertaken, we also noted 
areas where processes could be improved. 


Bridge Inspection and Maintenance 
Ontario has about 14,800 bridges. The Ministry of 
Transportation is responsible for ensuring that the - 
approximately 2,800 bridges located within the 
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provincial highway system are safe. Municipalities 
are responsible for the other 12,000 bridges. There 
is a legislative requirement that all bridges must 
be inspected every two years, using the detailed 
inspection standards established by the Ministry. 
Although the Ministry was conducting these inspec- 
tions every two years, we noted weaknesses in the 
oversight and follow-up of inspection results. In 
addition, to conduct the required close-up inspec- 
tions, lane closures are sometimes necessary. How- 
ever, there have been no requests for lane closures 
whatsoever for the last three years in the Greater 
Toronto Area. As well, the Ministry does not have 
the authority to assess whether municipalities are 
inspecting their bridges every two years in accord- 
ance with the required inspection standards. 


Infection Prevention and Control 

at Long-term-care Homes 
There is a high risk of infectious diseases, such 
as C. difficile, spreading among residents of long- 
term-care homes. This is because residents often 
share rooms and participate in activities together 
and older people are generally more vulnerable to 
illness. At the long-term-care homes we visited, we 
noted that a number of processes had been insti- 
tuted to prevent and control the spread of infectious 
diseases. However, improvements were required 
with respect to the screening of new residents, 





ensuring that resident rooms were being properly 
cleaned, and monitoring the use of antibiotics to 
reduce the incidence of C. difficile. 


Consumer Protection 
The Ministry of Consumer Services oversees busi- 
ness practices for the protection of consumers 
and the ensuring of public safety. Our independ- 
ent external consumer survey indicated that the 
‘Ministry needs to better promote its mandate and 
the services it makes available to consumers. As 
well, it needs to be more proactive in overseeing 
problem industries and repeat offenders. Although 
the Standing Committee on Public Accounts had 
recommended in 2004 that the Ministry conduct 
a review of how well industry-sponsored oversight 
authorities were protecting the public, it was only 
after a tragic propane explosion in August 2008 
that the Ministry launched a comprehensive review 
of this area. 


Social Housing 
Social housing is rent-subsidized accommodation 
provided to people who, without such financial 
support, would have difficulty putting a roof over 
their heads. Social housing has been primarily a 
municipal responsibility since 2000. However, from 
both a value-for-money perspective and from the 
perspective of those who live there, it is critical that 
the province monitor whether such housing is being 
maintained in a reasonable and safe condition. 
Given that the average age of the social-housing 
stock is about 40 years, the deteriorating condition 
of many of the housing units has been a significant 
and growing concern. Furthermore, we found that 
the Ministry had no information on the actual con- 


dition of the social-housing portfolio across Ontario. 


Teletriage Health Services 

Ontario’s teletriage services (comprised of Tele- 
health Ontario and the Telephone Health Advisory 
Service) provide callers with free, confidential tele- 
phone access to a registered nurse. Although our 
independent survey indicated that those who used 
Telehealth Ontario were generally satisfied, only 

a small portion of Ontario’s population used the 
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services. Furthermore, there are improvements that 
could be made to enhance the services provided, 
such as ensuring that newly hired nurses have 

the required clinical experience and conducting 
independent reviews of the quality of the advice 
provided by nurses. 


THE OFFICE’S FINANCIAL AUDIT 
RESPONSIBILITIES 


In addition to value-for-money auditing, we audit 
the province’s consolidated financial statements 
and the financial statements of numerous Crown 
agencies to “close the accountability loop” and 
ensure that the Legislature and the people of 
Ontario receive credible financial information 
about the state of public finances. We discuss our 
work relating to our audit of the province’s financial 
statements in Chapter 2. The Crown agencies we 
either directly audit or which are audited by other 
accounting firms under our direction are listed in 
Exhibit 1. 

Two of the more significant observations dis- 
cussed in greater detail in Chapter 2 this year are as 
follows: 

e For the 16th straight year, we were able to 
report that the province’s financial statements 
were presented fairly, in accordance with 
generally accepted accounting principles for 
governments in Canada. This means that 
legislators and the public can have confidence 
that the province’s financial condition and 
annual operating results as reflected in those 
statements is a fair portrayal of the govern- 
ment’s management of the “public purse” over 
the past fiscal year. © 

e As discussed previously, the Workplace and 
Safety Insurance Board (WSIB) has a growing 
and significant unfunded liability of $11.5 bil- 
lion. Because the WSIB is currently classi- 
fied as a trust for accounting purposes, this 
unfunded liability and its annual operating 
results are not included in the consolidated 
results of the province. However, given the 
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size of the unfunded liability and the fact 
that the government controls the WSIB under 
current accounting standards, we questioned 
whether the WSIB meets the definition of a 
true trust. If it does not, it should be included 
in the province’s consolidated financial state- 
ments. We recommended the Ministry of 
Finance formally assess this issue. 


OTHER WORK DONE DURING THE YEAR 


Each year, we follow up on actions taken to imple- 
ment our recommendations from value-for-money 
audits published two years earlier. The results of 
this follow-up work are in Chapter 4. In a number of 
instances, we noted that considerable progress had 
been made in addressing our recommendations. 

We also fulfilled our responsibilities this year 
under the Government Advertising Act, 2004, as 
discussed in Chapter 5. This Act requires us to 
review proposed government advertising intended 
for television, radio, newspapers, magazines, and 
billboards, as well as items to be delivered to house- 
holds by bulk mail delivery, to ensure that they | 
do not have as a primary objective the promotion 
of the partisan political interests of the governing 
party. | 

The Legislature’s all-party Standing Committee 
on Public Accounts held hearings on a number 
of value-for-money audits published in our 2008 
Annual Report. The Committee’s work enhances the 
accountability of ministries, agencies, and broader- 
public-sector organizations to the Legislature and 
the citizens of this province. An overview of the 
Committee’s work is presented in Chapter 6. 


Summaries of Value-for- 





money Audits and Reviews 


The following are summaries of the value-for- 
money audits and the review reported in Chapter 3 
of this Annual Report. 


3.01 ASSISTIVE DEVICES PROGRAM 


The Ministry of Health and Long-Term Care (Min- 
istry) administers the Assistive Devices Program. 
Its objective is to provide support and funding to 
Ontario residents with long-term physical dis- 
abilities to assist them in obtaining personalized 
assistive devices that enable them to function more 
independently. In the 2008/09 fiscal year, program 
expenditures were approximately $347 million, and 
the Program supported about 294,000 clients. 
Although the Ministry has improved service deliv- 
ery to clients since our last audit of the Program, it 
should manage payments more economically and 
enforce eligibility and other policy requirements 
more rigorously. Some of our more significant 
observations were as follows: 

e A majority of people getting oxygen at home 
use oxygen concentrators that cost between 
$400 and $1,000 and last five to seven years. 
These devices also require periodic servicing. 
The Ministry typically pays vendors about 
$23,000 over a five-year period for the pur- 
chase and servicing of these devices without 
analyzing whether the actual servicing costs 
incurred by the vendor support such a price 
differential. 

e Vendors’ rates of return in selling assistive 
devices were higher than the targeted return 
of 33% set by the Ministry. Average vendor 
mark-ups for mobility devices, respiratory 
devices, and computer systems were 84%, 
117%, and 128%, respectively. In setting 
device prices, the Ministry had not taken into 
account the significant price decreases arising 
from technological advances and the potential 
for some vendors to obtain volume discounts. 

e Vendors are receiving even greater returns 
from computer components such as monitors, 
printers, and scanners. For example, the 
Program-approved price for a monitor that 
often costs vendors only about $250 is $1,332, 
giving vendors a potential return of 400%. We 
also found that vendor price quotes for the 


same computer system varied significantly 
and ranged from $1,300 to $4,400. 

The Ministry was not consistently monitoring 
scooter claims to identify unusual patterns, 
nor was it taking appropriate action to prevent 
potential abuses. We noted that scooter claims 
of some vendors increased by more than 
800% over the last three years. 

In our sample, one-third of the assessments 
that ought to have been conducted by oxygen 
vendors to confirm clients’ continued eligibil- 
ity for home oxygen either had not been done 
or showed that the clients no longer met the 
criteria for long-term home oxygen supply. Yet 
the Ministry was not made aware of this and 
continued to pay for their home oxygen. 
Claims for Frequency Modulated (FM) hear- 
ing devices have risen dramatically among 
seniors, increasing from $250,000 in 2004/05 
to $4.8 million in 2008/09. However, some 
clients indicated that they did not really need 
or use the FM systems. 


e We noted cases where individual author- 


izers—health-care professionals who can 
authorize the subsidized purchase of assistive 
devices— were signing more than 90% of 
certain vendors’ claims. One such vendor had 
claimed more than $10 million for hearing 
aids since 2000. Some authorizers continually 
referred clients to the same vendors, even if 
there were other vendors located much closer 
to where the client lived. The Ministry knew 
about some of these cases for several years yet 
took no remedial action. 

Ontario does not recycle used manual wheel- 
chairs to lower the impact on the environment 
or save on costs. Other provinces, such as 
Alberta and Quebec, have achieved cost sav- 
ings of $4 million to $5 million per year from 
recycling their manual wheelchairs. 
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3.02 BRIDGE INSPECTION AND 
MAINTENANCE 


Ontario has about 14,800 bridges. Approximately 
12,000 of these are located in municipalities 

and are the responsibility of municipalities. The 
remaining approximately 2,800 bridges are located 
within the provincial highway system and are the 
responsibility of the Ministry of Transportation 
(Ministry). The average age of the province’s bridge 
infrastructure is about 40 years. Under the Public 
Transportation and Highway Improvement Act, all 
provincial and municipal bridges are required to 
be inspected every two years under the direction 
of a professional engineer, in accordance with the 
Ministry’s Inspection Manual. 

We noted that the Ministry was conducting the 
inspections every two years as required. We also 
noted that the Ministry had established compre- 
hensive standards for bridge inspection in Ontario 
and that its standards had been adapted for use by a 
number of other Canadian jurisdictions. 

However, we did find a number of areas where 
improvements to the Ministry’s inspection and 
maintenance processes would help minimize safety 
risks and ensure that the bridges the province is 
responsible for remain safe. Our specific observa- 
tions were as follows: 

e According to the Ministry’s assessment, more 
than 180 provincial bridges (7% of the total) 
were in poor condition, requiring repair or 
rehabilitation work within one year of inspec- 
tion. However, we found that over one-third 
of these bridges were not included in the 
Ministry’s capital work plan for the upcoming 
year. 

e@ The Ministry had not ensured that informa- 
tion in its bridge inventory database with 
respect to critical elements of each bridge 
was accurate and complete. In addition, the 
database did not have information on the 
rehabilitation history for almost one-third of 
bridges 40 or more years old. 
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e The Inspection Manual requires a detailed 
visual “close-up” inspection of each bridge 
element. Normally, this requires the clos- 
ure of lanes and road shoulders to traffic. 
Without closing a lane, close-up inspection 
of the critical elements of certain bridges on 
Highway 401 in the Greater Toronto Area, for 
example, is not possible, yet there have been 
no such lane closures for the past three years. 

e We found several weaknesses in the process 
for ongoing oversight of inspections. For 
example, although the Inspection Manual 
stipulates that a typical bridge inspection take 
two to three hours, we found on 36 occasions 
between 2006 and 2008 that 10 or more 
bridges were inspected in one day. In other 
instances, we noted an improvement in the 
overall condition rating of over 300 bridges, 
even though little or no rehabilitation work 
had been done on those bridges since the last 
inspection. 

e@ We noted that many of the maintenance rec- 
ommendations resulting from biennial bridge 
inspections were not carried out. In two of the 
three regions that we visited, only about one- 
third of the recommended maintenance work 
was actually completed, and the third region 
did not track this work at all. 

With respect to municipal bridges, there is cur- 
rently no legislation that requires or even enables 
the Ministry of Transportation or any other provin- 
cial ministry to oversee municipalities’ compliance 
with the required biennial inspections. There is 
no central database on the number of municipal 
bridges and the overall condition of these bridges. 
In response to a survey we conducted, 85% of 
municipalities indicated that they had a backlog 
of rehabilitation work. Municipalities with a large 
number of bridges relative to their population 
and revenue base had more difficulty funding the 
rehabilitation of bridge infrastructure and therefore 
had more significant backlogs. 


3.03 CONSUMER PROTECTION 


The Ministry of Consumer Services (Ministry) over- 
sees business and industry practices in Ontario’s 
consumer marketplace for the protection of con- 
sumers and public safety. It does this by establish- 
ing a regulatory and legal environment aimed at 
protecting consumers, educating the public and 
businesses, addressing complaints from the public, 
monitoring and inspecting businesses, and enfor- 
cing compliance with such consumer-protection 
regulations and laws as the Consumer Protection 
Act, 2002. 

Eight designated administrative authorities, 
which are not-for-profit corporations, are respon- 
sible for a number of consumer and public-safety 





statutes relating to specific marketplace sectors. 
The Ministry monitors the performance and activ- 
ities of these delegated authorities. 

In 2008/09, the Ministry carried out these 
responsibilities with a staff of about 110 and operat- 
ing expenditures of approximately $12.6 million. 

The Ministry has made progress in addressing 
many of the recommendations in our 2003 audit, 
including recently improving its oversight of dele- 
gated authorities. Several changes to legislation 
have also strengthened consumer protection, and 
the Ministry has carried out initiatives to promote 
compliance with consumer-protection legislation by 
certain industries. Nonetheless, we noted that cor- 
rective action is required in the following areas: 

e The Ministry needs to better promote its man- 
date and services to consumers. The almost 
40,000 inquiries and written complaints it 
received in the 2008/09 fiscal year represent 
a 12% drop in volume from peak levels in 
2004/05, but the Ministry has done no work 
to assess the reasons for this decline. Quebec’s 
consumer protection agency, by comparison, 
receives more than 250,000 consumer inquir- 
ies and complaints annually. In addition, our 
own independent external survey indicated 
that the Ministry would not be among Ontar- 
ians’ top choices for resolving a complaint. 





e The Ministry needs to deal more effectively 


with problem industries and repeat offenders, 
such as collection agencies, which have con- 
sistently been on the Ministry’s Top 10 Com- 
plaints list from 2000 to 2008. As well, limited 
staff inspection resources resulted in no 
proactive visits during 2008/09 to the types of 
businesses in the Top 10 Complaints list, and 
the Ministry initiated only 148 inspections and 
educational field visits as a direct result of the 
6,000 written complaints it received. In addi- 
tion, the Ministry’s lack of inspection powers 
under the Consumer Protection Act, 2002, 
which covers most businesses, hinders it from 
identifying consumer-protection violations. 
The Ministry made some progress since our 
last audit in enforcing compliance by cem- 
etery owners with reporting requirements 
under the Cemeteries Act. However, we identi- 
fied a number of financial discrepancies that 
the Ministry had not followed up on. 

The Ministry launched a comprehensive 
review of delegated authorities on an urgent 
basis only after a tragic propane explosion in 
Toronto on August 10, 2008. However, the 
Standing Committee on Public Accounts had 
recommended four years earlier that the Min- 
istry conduct such a review. 

The boards of directors of delegated author- 
ities were dominated by representatives of the 
industries they regulate. The Ministry has not 
encouraged a greater balance of representa- 
tion from government, consumers, the public, 
and industry on such boards. 

The Ministry has no right to access delegated 
authorities’ information on matters such as 
quality-assurance programs, strategic plans, 
executive salary and compensation packages, 
and board minutes. 

We noted that only one performance measure 
is reported publicly to cover all consumer- 
protection programs delivered directly by the 
Ministry, and we questioned whether it was a 
reliable and meaningful measure. 
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3.04 EDUCATION QUALITY AND 
ACCOUNTABILITY OFFICE 


The Ontario government established the Education 
Quality and Accountability Office (EQAO) as a 
Crown agency in 1996, with a mandate to develop 
and report on province-wide tests of student 
achievement. Such assessment results are intended 
to provide reliable and objective data that the 
Ministry of Education (Ministry) and the province’s 
72 school boards can use to plan ways of improving 
student learning. 

Each year, the EQAO tests students in all Ontario 
publicly funded schools in Grades 3, 6, 9, and 10. 
Grade 3 and Grade 6 students are tested in read- 
ing, writing, and mathematics. Grade 9 students 
are tested only in mathematics. To graduate from 
high school, all students, including those in private 
schools, must pass the Ontario Secondary School 
Literacy Test (OSSLT), which is usually written in 
Grade 10. The EQAO spent $31.7 million in the 
2008/09 fiscal year, all of it funded by the Ministry. 

We found that the EQAO had adequate pro- 
cedures and controls for ensuring that its tests 
accurately reflected the Ministry’s curriculum 
expectations. We also noted that the EQAO, to 
ensure that the tests’ level of difficulty was compar- 
able between years, imposed strict criteria for the 
development and field-testing of questions, and 
thoroughly reviewed test content. The general con- 
sensus among stakeholders, including principals 
and teachers, was that the tests were generally 
an accurate reflection of students’ achievement in 
meeting the curriculum expectations. However, 
there are areas where improvements can be made: 

@ The EQAO hires an external contractor to visit 

selected schools to review pre-test prepara- 
tion, ensure test booklet security, observe the 
administration of the tests, and undertake 
other quality-assurance procedures. Overall, 
the external contractor has reported a high 
degree of compliance with EQAO adminis- 
trative procedures, but an improved school 
selection process is required to reduce the 
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risk of student cheating and non-compliance 
with administrative procedures. For example, 
10 of the province’s 72 school boards had not 
received a visit from the external contractor 
over the past five years with respect to the 
Ontario Secondary School Literacy Test— 
while a number of private schools with as few 
as five students taking this test were visited. 
The EQAO must ensure that the up to 1,700 
markers it hires and trains are consistent 
when grading test papers. To do so, it seeds 
“validity papers” (test papers previously 
scored by an expert panel) among the regular 
papers. The grades the markers give these 
validity papers are monitored to determine if 
retraining is required. The EQAO consistently 
meets its target of having 95% of the validity 
papers graded within one scoring level of 

the expert panel’s score. However, on a per 
question basis, the EQAO often does not meet 
its target of 70% of the questions having the 
same grade as that given by the expert panel. 
The EQAO informally reviews results at the 
school and school board levels. However, 
formal analysis and follow-up may be required 
to ensure that the testing process can be used 
more effectively to evaluate the reliability of 
assessment results. For example, we noted 
that some schools’ EQAO results fluctuated 
by as much as 50% from one year to the next, 
but these instances were not being systematic- 
ally flagged for follow-up to determine what 
accounted for such a dramatic change. 

The EQAO annually reports student test 
results as well as results from questionnaires 
given to students, teachers, and principals on 
its activities. The school staff we interviewed 
stated that the questionnaires did not suffi- 
ciently allow for feedback on ways to improve 
the testing process. They also felt that the 
EQAO should take a bigger role in explaining 
the assessment process to parents and other 
stakeholders. 


3.05 GOVERNMENT USER FEES 


Ontario collected almost $2.2 billion in user fees in 
2008/09, which represents about 2% of total prov- 
incial revenues. Compared to most other provinces, 
Ontario collects less in terms of percentage of total 
revenues obtained from user fees and user fees 
charged on a per capita basis. 

A user fee is generally charged to recover all or 
a part of the costs of providing a specific good or 
service, such as a vehicle registration, to the indi- 
viduals and businesses that request it. In contrast, a 
tax is used to produce revenues for general govern- 
ment purposes and for goods and services that the 
government deems to be a “public good,” such as 
health care. 

The Ministry of Transportation collects almost 
half of all user-fee revenues, for driver’s and carrier 
licences and vehicle registrations. The Alcohol and 
Gaming Commission of Ontario collects another 
22% for liquor licences and permits. 

A 1998 Supreme Court of Canada decision 
concluded that user fees could be considered 
unlawful and therefore may be repayable if they 
were determined by a court to be a tax that was 
not established by enacted legislation or if the fee 
amounts charged were excessive and did not have 
a reasonable relationship to the cost of the services 
provided. Although the Ontario government has 
taken some action over the past decade to address 
this ruling, there are still user-fee revenues col- 
lected by the Alcohol and Gaming Commission 
and the Ministry of Government Services of over 
$500 million annually that may be at risk because 
they may not fit the Supreme Court’s criteria for 
valid fees. 

The Non-Tax Revenue Directive, established in 
1991, is intended to maximize the Ontario govern- 
ment’s non-tax revenues, including user fees, and 
ensure that ministries regularly review services 
and rates, and keep non-tax revenue rates up to 
date. However, we found that the existing processes 
were, for the most part, not effective in achieving 
the Directive’s goals. In addition, unlike user-fee 





legislation in place federally and in some other 
provinces, Ontario’s existing policies and proced- 
ures lack transparency and public involvement in 
key decisions about changes to user-fee rates, nor is 
there sufficient public reporting on fees collected, 
their use, and the costs associated with providing 
the fee-related services. 

A key principle of the Directive is that, when 
it is reasonable and practical to do so, the cost of 
providing services to the public should be borne 
by those who benefit from the service. In 2008, as 
part of the Budget process, the Ministry of Finance 
conducted a one-time review, which indicated 
that forecasted user-fee revenues would recover 
less than 75% of the costs identified for these fee- 
related services. In cases where a ministry decides 
not to charge the full cost of a service—such as 
when it is not practical or economical to do so, or 
users cannot afford to pay—the Directive requires 
that the ministry document the reasons for setting 
fees at reduced rates. We noted that, for the most 
part, this was not being done. 

In addition, there were generally no recurring 
processes in place to keep user-fee rates up to date, 
as is required under the Directive. We noted many 
examples of fees that have had no rate increase for 
10 to 20 years, despite the fact that the fees were set 
at amounts that recovered only from 23% to 45% of 
the full costs of providing the services. 

Ministry of Finance guidelines require that 
ministries discount fees for services provided elec- 
tronically to encourage their increased use by the 
public. We noted that no discounts were offered by 
the Ministry of Transportation for driver’s licences 
and vehicle registrations via the Internet or at elec- 
tronic kiosks. On the contrary, services at electronic 
kiosks across the province incur a so-called “con- 
venience” surcharge of one dollar per transaction. 
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3.06 INFECTION PREVENTION AND 
CONTROL AT LONG-TERM-CARE HOMES 


Long-term-care homes, such as nursing homes 

and charitable homes, provide care, services, and 
accommodations to individuals unable to live 
independently and requiring the availability of 
24-hour care. There are more than 600 such homes 
in Ontario, caring for about 75,000 residents, most 
of whom are over 65 years old. In the 2008/09 
fiscal year, funding to long-term-care homes by the 
Ministry of Health and Long-Term Care (Ministry) 
through the Local Health Integration Networks 
totalled $2.8 billion, with residents generally also 
making a co-payment of between $1,600 and 
$2,200 per month for accommodation. 

There is a high risk of infectious diseases [such 
as Clostridium difficile (C. difficile), febrile respira- 
tory illness (FRI), methicillin-resistant Staphylo- 
coccus aureus (MRSA), and vancomycin-resistant 
enterococci (VRE)] spreading among residents of 
long-term-care homes because they often share 
rooms and generally eat and participate in activities 
together. As well, older residents are generally more 
vulnerable to illness. When a resident acquires an 
infection in a home, it is considered a health-care- 
associated infection (HAI). 

All three long-term-care homes we visited— 
Extendicare York in Sudbury, Nisbet Lodge in 
Toronto, and Regency Manor in Port Hope—had a 
number of processes in place to prevent and control 
HAIs. Furthermore, these homes had all recently 
conducted their first review of staff compliance 
with certain hand-hygiene policies, since hand 
hygiene is the most important activity for control- 
ling the spread of infections. However, we noted 
areas where these homes could improve their prac- 
tices. Some of our more significant observations 
included the following: 

e@ The Ministry does not have information on the 
total number of cases of HAIs in long-term- 
care homes. The information collected at the 
homes we visited was generally not compar- 
able because the homes defined and counted 
HAIs in different ways. 
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Although the homes visited had policies to 
screen new residents for FRIs, documentation 
at two of the homes indicated that just 60%— 
80% of new residents sampled were screened. 
At the third home, there was no evidence of 
formal screening for FRIs. 

Each home had a policy to test new residents 
for tuberculosis within 14 days of admission, 
as required by legislation. One home tested 
all new residents in our sample, but the other 
two tested only 70% and 80%, and much later 
than required. 

Homes generally did not have unoccupied 
rooms to move infectious residents into. 
Although the Provincial Infectious Diseases 
Advisory Committee (PIDAC) recommends 
cleaning the rooms of residents who have C. 
difficile twice a day, none of the homes did 
this. 

In the 2008/09 fiscal year, 81 C. difficile 
outbreaks in homes were reported to the 
Ministry. The increased use of antibiotics has 
been shown to increase the risk of C. difficile. 
None of the homes had a formulary that lists 
the antibiotics that physicians can prescribe, 
as recommended by PIDAC. 

Unlike hospitals, long-term-care homes are 
not required to report publicly on certain 
patient-safety indicators, such as health-care- 
acquired cases of C. difficile, MRSA, and VRE, 
as well as hand-hygiene compliance among 
health-care workers. 

None of the Infection Prevention and Control 
Professionals designated by the homes had 
the specific training recommended by PIDAC. 


3.07 LITERACY AND NUMERACY 
SECRETARIAT 


The Ministry of Education (Ministry) is responsible 
for the system of publicly funded elementary and 
secondary school education in Ontario. Its respon- 
sibilities include developing the primary and sec- 
ondary school curricula, setting requirements for 
student diplomas, and providing funding to school 
boards. The Ministry also set up the Education 
Quality and Accountability Office (EQAO)—a gov- 
ernment agency—to provide independent assess- 
ments of student achievement by testing students in 
reading, writing, and mathematics. The Ministry’s 
Literacy and Numeracy Secretariat (Secretariat), 
the subject of this audit, was established in Nov- 
ember 2004 to help more than 4,000 elementary 
schools across 72 school boards to meet student- 
achievement targets. Since it was established in 
2004, the Secretariat has spent $340 million, with 
almost $288 million transferred to school boards. 

The Ontario government made a significant 

commitment to improving student achieve- 

ment when, in 2004, it set a goal that 75% of all 
12-year-olds (grade 6 students) would score a 
level-three standard (approximately a B average) 
on province-wide testing for reading, writing, and 
mathematics by 2008. Although the Ministry had 
not achieved this goal by 2008, substantial progress 
has been made over the last five years, and the 
percentage of 12-year-olds achieving the provincial 
standard increased from 56% in 2003/04 to 65% in 
2007/08. Further increasing this percentage will be 
a challenging undertaking, but we noted a number 
improvements that could be made to help achieve 
this goal. Some of our more significant observations 
were: 

e Although the Secretariat and the school 
boards we visited have done some limited 
assessment of the effectiveness of secretariat 
programs, further analysis is required if the 
Secretariat is to ensure that its spending of 
almost $288 million has been directed to the 
initiatives that provide the most benefit. 
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e School board improvement plans were initi- 3.08 ONTARIOBUYS PROGRAM 


ated to help teachers, principals, and school i ; Seay ; 
OntarioBuys is a government initiative launched in 


2004 to help the broader public sector (BPS) save 
money when procuring goods and services. 

Since 2004/05, OntarioBuys has provided 
funding of about $148 million for two areas: about 


board staff plan and implement strategies to 
improve student achievement. The Ministry 
has developed a framework for an effective 
improvement planning process. However, 


neither the Secretariat nor the boards we Bie : : 
AS ; $88 million for the formation and/or expansion 
visited documented, monitored, or reported : f 
of collaborative purchasing groups called “shared- 
on the plans to the extent necessary to assess : ogee 
a service organizations” (SSOs) whose members are 
whether the plans were contributing to ey Stk) IY if 3 
BPS institutions, and $61 million for 53 projects 
improved student achievement. Also, because ; i pitta 
Die, aimed at helping BPS institutions become more effi- 
it exercised only limited oversight, the Sec- : ee ; ; 
Pann ; cient and effective in their supply-chain and other 
retariat did not have the information needed 
back-office processes. 

to identify patterns and trends among school NG 
The government announced in its March 2009 


Budget that OntarioBuys had helped BPS entities 
redirect $45 million in savings toward front-line 


boards, so it could not identify the most suc- 
cessful initiatives and share them with other 


boards. 
: services. We found, however, that almost $20 mil- 
® Secretariat program funding was not always irah 
} lion of this reported amount was not redistributed 
allocated to school boards and schools with ; j 
; ; to hospitals to provide front-line services but 
the greatest need. Rather, funding allocation 
rather was retained by the SSO that generated the 
was based on average daily enrollment or the ; 
; ; savings to develop information technology for its 
reason a given amount of funding went to a ; 
back-office processes. The balance of the reported 
school board could not be fully explained by 
savings came from a number of projects; however, 
the Secretariat. For instance, we found that, i ; ; i 
OntarioBuys did not verify these savings nor was 
for one major program, the board with the 
it able to demonstrate that they had actually been 
greatest number of schools designated as 
invested in front-line services. 
low-performing received only $17 per student, 
: , Although OntarioBuys has undertaken 
while several boards with no schools desig- as : } 
; significant efforts to promote its collaborative 
nated as low-performing received more than = ou tase bint vistes 
supply-chain initiatives, participation in the SSOs, 
twice this amount per student. 
particularly in the education sector, is currently 
@ The Secretariat routinely uses certain boards 
i ‘ Nii? well below the level required for OntarioBuys to 
as “bankers” to act as distributors of funds ; j eee 
} achieve its goals. Our other significant observations 
to third parties or other school boards. We Risen 
included: 
questioned the need for such arrangements ens aise 
} e OntarioBuys approved funding for projects 
and noted that there is no Memorandum of u PP : Saale 
on the basis of business cases prepared by 


BPS organizations that included estimated 
costs and potential savings. However, the 


Understanding or agreement between the 
Secretariat and the banker boards outlining 
respective roles and responsibilities, account- 
i ; reasonableness of the estimates were often 
ability relationships, reporting requirements, 

not appropriately assessed. For example, the 
largest project approved for funding projected 
savings of $669 million over five years, but we 
found that $294 million of this amount was 


not included in OntarioBuys’ funding review 


and service levels to be provided. Also, the 
Secretariat paid banker boards administrative 
fees that in some cases appeared excessive. 
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and that the balance of $375 million was 
determined on an arbitrary basis. Subsequent 
to our audit fieldwork, OntarioBuys revised 
the estimated savings down to $113 million 
over five years. 

The education SSO, which has received 

$30 million in OntarioBuys funding since 
2005, committed to sign up 13 of the prov- 
ince’s educational institutions and 1,000 
suppliers by June 2009 to participate in a new 
electronic purchasing system called e-Market- 
place. As of June 2009, e-Marketplace had yet 
to become operational and no institutions had 
formally signed up to be members. 

We reviewed a list of project savings that 
OntarioBuys provided us and found some 
purported savings to be questionable. For 
example, our review of savings totalling 

$7.3 million for two projects, which were 
supposed to be completed by December 2006, 
showed that neither project had been com- 
pleted by the time of our audit. Subsequent to 
our review, OntarioBuys revised the total sav- 
ings for the two projects down to $1.1 million. 
OntarioBuys did not have program-specific 
guidelines for consistent and effective mon- 
itoring of project progress, with requirements 
for conducting site visits, documenting work 
performed, verifying deliverables prior to the 
release of final payments, and closing files for 
completed projects. 

Since the 2004/05 fiscal year, the SSOs and 
BPS organizations involved in the projects 
spent about $45 million of the funding pro- 
vided to them to hire some 270 consultants for 
a variety of reasons. We reviewed $15 million 
of consulting contracts from various projects 
and found that over 40% did not comply with 
the competitive procurement requirements of 
the project funding agreements. 


3.09 ONTARIO DISABILITY SUPPORT 
PROGRAM 


The Ministry of Community and Social Services 
(Ministry) administers the Ontario Disability Sup- 
port Program Act (Act), which provides income and 
employment support to approximately 250,000 
individuals with eligible disabilities (as defined by 
the Act). Total annual Ontario Disability Support 
Program (ODSP) benefit payments have risen to 
more than $3 billion, a 42% increase since the time 
of our last audit in 2004. 

ODSP income support is intended to assist with 
basic living expenses such as food, shelter, clothing, 
and personal-needs items. Although employment- 
support programs are available to ODSP recipients, 





participation in them is not required. As a result, 
relatively few ODSP recipients join such programs. 

Although the Ministry had implemented a num- 
ber of the recommendations contained in our 2004 
Annual Report, serious issues remain in determining 
an applicant’s financial eligibility and the amount 
of assistance to be paid. 

The Ministry has established a two-stage process 
to ensure that only qualified applicants receive 
income support. The first stage relies solely on 
applicants volunteering financial information. To 
compensate for the risks associated with this, the 
second stage is third-party verification of certain 
information provided by the applicant. However, 
this verification is largely ignored in practice. As a 
result, the Ministry is not adequately ensuring that 
only eligible individuals receive the correct pay- 
ment amounts. We also noted the following: 

e Although the Ministry has significantly 
reduced the average wait time for a medical- 
disability determination decision, 60% of 
recipients sampled still received late pay- 
ments. On average, they experienced a 58-day 
delay after they had been determined to be 
medically qualified for payments, which is 
almost three times longer than the outside 
limit of 21 days established by the Ministry. 
These delays in receiving approved bene- 
fits offset to a significant degree the good 


progress made since our last audit in expedit- 
ing the initial medical determination. 
Oversight procedures are lacking to monitor 
and assess the fairness and consistency of 
decisions made by individual adjudicators 

at the Disability Adjudication Unit (DAU). 
Consequently, the rates at which adjudicators 
determined that applicants were eligible gen- 
erally varied from 11% to 49%. 

In the 2008/09 fiscal year, 55% of applicants’ 
appeals to the Social Benefits Tribunal led to 
the Ministry’s initial decision to not approve 
an applicant for benefits being overruled. 
Since 2002, the Ministry has not performed 
any of the periodic medical reassessments— 
required by legislation—to ensure continuing 
eligibility for disability support payments. 
The Ministry relies on one individual to do all 
the assessment and reassessment work for any 
given file, yet the individual’s work is neither 
supervised nor reviewed to ensure that the 
decisions made comply with ministry and 
legislative requirements. 

The total amount of overpayments for both 
active and inactive accounts has increased 
substantially, from $483 million in 2004 to 
$663 million as of March 31, 2009. In many 
cases, overpayments resulted from what 
would appear to be recipients fraudulently 
misrepresenting their circumstances. These 
overpayments might often have been avoided 
if the Ministry had more effectively reassessed 
the eligibility and the amounts to be paid 

to those individuals whom its own systems 
identified as high-risk or followed up on tips 
received from the public. 

The Ministry’s computerized Service Delivery 
Model Technology (SDMT) information 
system still lacks key internal controls, and 
regional and local offices are not receiving, in 
an easily understandable format, the informa- 
tion they need to effectively oversee program 
expenditures. 
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3.10 ONTARIO RESEARCH FUND 


The Ontario Research Fund (Fund) was created in 
2004 to “support scientific excellence by supporting 
research that can be developed into innovative 
goods and services that will boost Ontario’s econ- 
omy.” It funds the direct and indirect operational 
costs of research through its Research Excellence 
Program, and the capital costs of research through 
its Large Infrastructure Program and Small Infra- 
structure Program. The Fund is the responsibility of 
the Ministry of Research and Innovation (Ministry), 
which was created in 2005 and focuses its efforts 
on activities that support Ontario’s knowledge 
economy and create high-value jobs. Since the 
Fund’s inception in 2004, and up to March 31, 
2009, expenses for the capital and operating com- 
ponents have amounted to $303 million, with total 
announced program commitments of $623 million. 

In our 2003 audit of the Science and Technol- 

ogy Division of the former Ministry of Enterprise, 
Opportunity, and Innovation, we reported signifi- 
cant concern over the lack of effective governance 
and accountability mechanisms. The consolidation 
of operating and capital research funding into one 
program managed and administered by the Min- 
istry has helped address these concerns. However, 
there are still a number of areas where improve- 
ments are required, noted as follows: 

@ The Fund’s overall mandate emphasizes sup- 
porting research that will provide economic 
and social benefits for the people of Ontario 
through the commercialization of research. 
However, most of the $623 million committed 
to research projects in the province funded 
basic (that is, theoretical) research that was 
not focused on commercial potential. 

@ The Ministry reports on how its programs are 
performing against three targets: the dollar 
value of investments made by the private sec- 
tor; the number of individuals with enhanced 
skills involved in Ministry-funded projects; 
and active licences for intellectual property 
rights that have resulted from Ministry-funded 
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projects. However, the Ministry does not 
measure or report publicly on the program’s 
contribution to the Ministry’s overall strategy 
of creating high-paying jobs and commercial- 
izing research. 

The Ministry did not have an adequate process 
to ensure that the projects funded through 
the Large Infrastructure Program supported 
Ontario’s strategic priorities or provided stra- 
tegic benefits to Ontario. In fact, the province 
funded $41.5 million worth of projects that 
did not directly support Ontario’s strategic 
priorities. Also, $65 million of program fund- 
ing under the Research Excellence Program 
was allocated to some very large projects for 
which it was questionable whether they met 
the program’s eligibility criteria. 

The Ministry relied on the the Canada 
Foundation for Innovation (CFI) to monitor 
Research Infrastructure Program grants and 
did not sufficiently assess or review the CFI’s 
work to ensure that the more than $300 mil- 
lion in program funding commitments to date 
were being spent for the approved purpose. 
Ontario’s colleges tend to focus on applied 
programs and research and helping small- 
to-medium-sized businesses develop tech- 
nologies and processes for the marketplace. 
However, no funding has been awarded 
directly to Ontario’s colleges. The Ministry 
should assess the potential benefits of applied 
research projects that address both the 
unique needs of Ontario’s colleges and offer 
enhanced commercialization potential. 

As part of the monitoring process for the 
Research Excellence Program, the Ministry 
receives various reports from grant recipients. 
However, we found that the Ministry had not 
performed any formal monitoring or clarified 
its expectations for independent audits to ver- 
ify the information submitted by recipients to 
ensure that program funds were being spent 
for the intended purpose. 


3.11 ONTARIO WORKS PROGRAM 


Under the Ontario Works Act, 1997, the Ministry of 
Community and Social Services (Ministry) provides 
income and employment assistance to approxi- 
mately 200,000 individuals who are unemployed or 
underemployed. The income assistance is intended 
to help recipients with basic living expenses such as 
food, clothing, and shelter while the employment 
assistance includes a variety of activities designed 
to increase employability and help recipients obtain 
paid employment in order to become self-reliant. 
For the 2008/09 fiscal year, the Ministry’s Ontario 
Works expenditures totalled almost $1.9 billion— 
$1.5 billion for income assistance, $171 million 

for employment assistance, and $194 million for 
program administration. 

The Ontario Works program is delivered on 
behalf of the Ministry by 47 Consolidated Munici- 
pal Service Managers and District Social Services 
Administration Boards as well as 100 First Nations, 
all referred to as service managers. A service 
manager is typically either a large municipality or a 
grouping of smaller ones, and each is accountable 
to one of the Ministry’s nine regional offices. 

Although the Ministry had implemented a 
number of the recommendations contained in our 
last audit of the program in 2002, there has been 
limited improvement in the overall administration 
of the program since that time. It remains our view 
that the Ministry still has inadequate assurance that 
only eligible individuals are receiving the correct 
amount of financial assistance. 

Our more significant concerns about the Min- 
istry’s oversight of Ontario Works program delivery 
by the service managers included the following: 

e During the Ontario Works application pro- 
cess, service managers relied on individuals 
to provide almost all of the information used 
to assess their eligibility for assistance and 
seldom undertook the required third-party 
verifications designed to help assess the 
completeness and accuracy of the information 
provided by applicants. 
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e Benefits for such things as community and 
employment start-up activities were often 
paid without any evidence that the activity 
had occurred and in amounts that exceeded 
the established maximums. 

e Many special dietary allowances were paid 
under questionable circumstances. The total 
amount spent on the allowances has increased 
from $5 million in the 2002/03 fiscal year to 
more than $67 million during 2008/09. 

e Unrecovered overpayments to about 350,000 
current and former Ontario Works recipients 
increased 45%, from $414 million in February 
2002 to $600 million as of March 31, 2009. 
Efforts by service managers to recover these 
overpayments have been minimal. 

e Many tips from the fraud hotline were either 
inadequately investigated or ignored. 

e@ The Ministry did not have enough informa- 
tion to assess whether employment assistance 
funds were being used as intended and were 
helping people obtain employment. 

e@ The Ministry’s examination of a sample of 
service managers’ reimbursement claims 
for the Ministry’s share of program costs did 
not occur annually as required, nor did the 
examinations ensure that submitted claims 
were complete, accurate, and based on actual 
payments made to assistance recipients. 

Despite improvements to the Ministry’s Service 

Delivery Model Technology information system 
(SDMT), which has been used by service managers 
to deliver the Ontario Works program since 2002, 
there continue to be concerns about the system’s 
reliability and its known deficiencies. 


3.12 SOCIAL HOUSING 


Social housing is rental accommodation developed 
with government assistance for a range of low- and 
moderate-income households, including families 
with children, couples, singles, and seniors. It can 
be owned by governments, as in the case of public 
housing, or by non-profit or co-operative organiza- 
tions. In Ontario, households in social housing that 
receive a subsidy to help pay rent typically pay a 
maximum rent equal to about 30% of their total 
pre-tax income. 

Most social housing in Ontario was built 
between the mid-1960s and the mid-1990s by 
the Canada Mortgage and Housing Corpora- 
tion (CMHC) and the provincial government. In 
December 2000, the province passed the Social 
Housing Reform Act, 2000 (Act), which required 
municipalities to assume responsibility for social- 
housing programs previously administered by the 
Canada Mortgage and Housing Corporation and 
the province. The province designated 47 regional 
Consolidated Municipal Service Managers (Service 
Managers) to administer social-housing programs 
at the local level. As of the end of 2008, there were 
about 260,000 units of social housing in Ontario, 
consisting of 100,000 public-housing units and 
160,000 non-profit and co-operative units. 

From both a value-for-money perspective and 
from the perspective of those who live there, it is 
critical that social housing be maintained in good 
condition. As well, sufficient and affordable social 
housing can have a significant impact on the health 
and safety of those Ontarians who depend on sub- 
sidized housing for a place to call home. However, 
the Ministry of Municipal Affairs and Housing 
(Ministry) collects little information on how well 
the $40 billion in social-housing stock is being 
maintained or whether there is an adequate supply 
to meet local needs. Our observations included: 

e As of December 31, 2008, the number of 
households on waiting lists for social housing 
across the province totalled about 137,000. 

In many urban cencres, the average wait time 
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to secure accommodation was more than five 
years—and one municipality had reported a 
wait time of 21 years for all categories of ten- 
ants except seniors. 

The deteriorating condition of social-housing 
stock—particularly the public-housing port- 
folio, whose units are an average of 40 years 
old—has been a significant and growing con- 
cern for municipalities. In 2006, for instance, 
the Toronto Community Housing Corporation 
identified immediate capital-repair needs of 
$300 million for its 60,000 public-housing 
units. However, the Ministry had no up-to- 
date and reliable information on the overall 
condition of the social-housing stock on a 
province-wide basis. 

A large number of the federal government’s 
funding agreements with housing providers 
will start to expire in 2015, with no guarantee 
that they will be renewed. Without continued 
funding, some existing social-housing projects 
will not be financially viable, even though 
Service Managers will still be required by law 
to maintain the prescribed minimum number 
of housing units. The Ministry had no firm 
plans to address Service Managers’ concerns 
regarding the possible ending of federal 
funding. 

In partnership with the federal government, 
Ontario has in recent years provided Service 
Managers with some additional funding for 
new housing programs. There was a general 
lack of reporting on the success of these 
programs. For example, although one such 
program did increase the supply of housing, 
the stipulated rent to be charged for more 
than half the units would not be considered 
affordable for households on, or eligible to be 
on, waiting lists. Better reporting and over- 
sight will be needed to effectively manage the 
significant new stimulus investments in social 
and affordable housing. 


3.13 TELETRIAGE HEALTH SERVICES 


Ontario’s teletriage services provide callers with 
free, confidential telephone access to a registered 
nurse for health-care advice and information. The 
services comprise Telehealth Ontario—available 
to all Ontario callers 24 hours a day, seven days a 
week—and the Telephone Health Advisory Service 
(THAS)—available Monday to Friday, 5 p.m. to 

9 a.m., and all day on weekends and holidays, 

to 8.4 million patients enrolled with physicians 
participating in various primary-health-care 
arrangements. 

The Ministry of Health and Long-Term Care 
(Ministry) contracts with a private service pro- 
vider to deliver the services. The service provider 
employs almost 300 registered nurses at its five call 
centres located throughout Ontario. The nurses 
use their clinical judgment in conjunction with 
medical-decision-support software to assist callers. 
During the 2008/09 fiscal year, 905,000 calls were 
responded to by nurses, and payments to the ser- 
vice provider totalled $35.1 million. 

We found that the Ministry had appropriately 
contracted for the delivery of teletriage services 
using a competitive process. The contract included 
a number of key performance requirements, mostly 
dealing with timely access to services, which 
the service provider reported that it met in the 
2008/09 fiscal year. Although only a small por- 
tion of Ontario’s population uses the services, our 
independent survey indicated that those who used 
Telehealth Ontario were generally satisfied. How- 
ever, we believe that improvements could be made 
to enhance the services. Our observations included: 

e Not only has the number of calls to teletri- 

age services been declining over the last few 
years, the number of calls as a proportion of 
the population is significantly less in Ontario 
than is the case in Alberta and Quebec. 
Furthermore, although over 60% of Ontarians 
were eligible to use THAS, only 1% of eligible 
individuals used it in 2008/09. 





e British Columbia and Quebec use the eas- 


ily remembered “811” phone number for 
their teletriage services, and certain other 
provinces are planning to adopt this phone 
number. Quebec reported a 15% increase in 
call volume following its implementation. At 
the time of our audit, Ontario had no plans to 
adopt the “811” phone number. 

The service provider indicated that about 25% 
of the callers in the live queue hung up before 
their call was answered. We calculated that 
85% of callers who waited spoke to a nurse 
within 23 minutes. Eighty-five percent of 
callers who left a call-back number spoke to a 
nurse within 34 minutes. 

Physicians who were on call to THAS had 

to be paged more than once in over 70% of 
calls requiring a page during 2008, and 9% of 
pages were never returned. 


e Although the service provider’s proposal, 


submitted to the Ministry in 2007, indicated 
that the service provider’s nurses would have 
at least three years of any type of nursing 
experience, 20% of our sample of nurses hired 
in 2008 had less than this. 

Unlike most provinces we spoke with, Ontario 
generally does not tape calls for subsequent 
quality assurance review. Rather, the ser- 

vice provider’s quality assurance reviewers 
sampled calls only as they were taking place 
and seldom did so during peak periods when 
nurses experience pressure to respond to wait- 
ing callers. The quality of advice was also not 
independently evaluated. 

In 2008/09, the Ministry paid the service pro- 
vider about $39 for each of the first 900,000 
registered calls to teletriage services and 
about $27 per call after that. Teletriage ser- 
vices costs for the three other provinces that 
shared cost information with us were about 
$20 per call. The Ministry had not determined 
the reason for the significant difference. 

The Ministry did not have performance stan- 
dards relating to the quality of nurses’ advice 
or how long callers wait in the live queue. 
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3.14 UNFUNDED LIABILITY OF THE 
WORKPLACE SAFETY AND INSURANCE 
BOARD 


The Workplace Safety and Insurance Board (WSIB) 
is a statutory corporation whose primary purposes 
are to provide income support and fund medical 
assistance to workers injured on the job. The work- 
place safety and insurance system receives no fund- 
ing from the government; it is financed through 
premiums charged on the insurable payrolls of 
employers. The government has the sole respon- 
sibility for setting benefits and coverage through 
legislation, while the WSIB has responsibility for 
setting premium rates. 

In our 2005 Annual Report, we noted that the 
assets in the WSIB’s insurance fund were substan- 
tially less than what was needed to satisfy the 
estimated lifetime costs of all claims currently in 
the system, thus producing what is known as an 
“unfunded liability,” which stood at $6.4 billion at 
that time. 

In our current review, we observed that, as of 
December 31, 2008, the unfunded liability stood at 
$11.5 billion, an increase of $3.4 billion from the 
previous calendar year. One factor that had a sig- 
nificant negative impact on the unfunded liability 
during 2008 was the global economic downturn. 

Failure to effectively control and eliminate the 
unfunded liability could result in the WSIB being 
unable to meet its existing and future financial com- 
mitments to provide worker benefits. Eliminating or 
reducing the unfunded liability requires that four 
key levers—legislated benefits, coverage, premium 
rates, and investments—work effectively in tan- 
dem. The WSIB and the government may have to 
commit to a different strategy with respect to these 
levers if the unfunded liability is to be eliminated 
within a reasonable period of time. 

Our other observations included the following: 

e@ The WSIB’s funding ratio of assets to liabilities 

was 53.5%, considerably lower than that of 
any of the four other large provincial boards 
we reviewed in British Columbia, Alberta, 
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Manitoba, and Quebec, which averaged 
102%. 

The WSIB and governments have sought over 
the last two decades to satisfy simultaneously 
two major stakeholders: employers, who 
wanted lower premiums, and workers, who 
wanted higher benefits. This has undoubtedly 
affected the size of the current unfunded 
liability. 

The WSIB’s ability to eliminate the unfunded 
liability has to some extent been limited by 
the government’s control over benefit changes 
and over which businesses and industries 

are covered by the system. For example, in 
Ontario, 72.6% of the workforce was covered 
by the system as of 2007, compared to 93.1% 
in British Columbia and 93.4% in Quebec. 


e Annual premium revenues in recent years 


have not been enough to cover benefit costs. 
Premiums have increased by an average of 
only 1% each year since 2001, at the same 
time as the WSIB was reporting average 
annual deficits of more than $900 million. 
Benefit and health-care costs have risen stead- 
ily over the last 10 years as a result of workers 
staying on benefits longer and receiving 
increases in those benefits as a result of legis- 
lative changes. 

The WSIB’s 15-year average rate of return on 
its investments from 1994 to 2008 was 6.6%. 
Given that future benefit costs are expected 
to rise at 7% annually, investments must earn 
more than 7% before any reduction of the 
unfunded liability can be realized solely from 
investment returns. 
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Introduction 





Ontario’s Public Accounts for each fiscal year end- 
ing on March 31 are prepared under the direction 
of the Minister of Finance, as required by the Min- 
istry of Treasury and Economics Act (Act). The Public 
Accounts comprise the province’s annual report, 
including the province’s consolidated financial 
statements, and three supplementary volumes of 
additional financial information. 

Preparing the consolidated financial state- 
ments is the Ontario government’s responsibility. 
This responsibility encompasses ensuring that the 
information, including the many amounts based 
on estimates and judgment, is presented fairly. The 
government is also responsible for ensuring that 
a system of control, with supporting procedures, 
is in place to provide assurance that transactions 
are authorized, assets are safeguarded, and proper 
records are maintained. 

My Office audits these consolidated financial 
statements. The objective of our audit is to obtain 
reasonable assurance that the statements are free 
of material misstatement—that is, that they are free 
of significant errors or omissions. The consolidated 
financial statements, along with my Auditor’s 
Report on them, are included in the province’s 
annual report. 

The province’s 2008/09 annual report also con- 
tains a Financial Statement Discussion and Analysis 
section that provides additional information 
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regarding the province’s financial condition and 
fiscal results for the year ending on March 31, 2009, 
including some details of what the government 
accomplished in the 2008/09 fiscal year. Providing 
such information enhances the fiscal accountability 
of the government to both the Legislative Assembly 
and the public. 

The three supplementary volumes of the Public 
Accounts consist of the following: 

@ Volume 1—statements from all ministries and 

a number of schedules providing details of the 
province’s revenues and expenses, its debts 
and other liabilities, its loans and investments, 
and other financial information. 

e Volume 2—audited financial statements of 
significant provincial corporations, boards, 
and commissions whose activities are 
included in the province’s consolidated finan- — 
cial statements, as well as other miscellaneous 
financial statements. 

@ Volume 3—detailed schedules of ministry 
payments to vendors and transfer-payment 
recipients. 

My Office reviews the information in the prov- 
ince’s annual report and in Volumes 1 and 2 of the 
Public Accounts for consistency with the informa- 
tion presented in the province’s consolidated finan- 
cial statements. 

The Act requires that, except in extraordinary 
circumstances, the government deliver its annual 
report to the Lieutenant Governor in Council on 
or before the 180th day after the end of the fiscal 


year. The three supplementary volumes must be 
submitted to the Lieutenant Governor in Council 


before the 240th day after the end of the fiscal year. 


Upon receiving these documents, the Lieutenant 
Governor in Council must lay them before the 
Legislative Assembly or, if it is not in session, make 
the information public and then, when the Legisla- 
tive Assembly resumes sitting, lay it before the 
Legislative Assembly on or before the 10th day of 
that session. 

This year, the government released the prov- 
ince’s 2008/09 Annual Report and Consolidated 
Financial Statements, along with the three Public 
Accounts supplementary volumes, on Septem- 
ber 25, 2009, meeting the 180-day deadline. 


The Province’s 2008/09 


Consolidated Financial 
Statements 





The Auditor General Act requires that I report 
annually on the results of my examination of the 
province’s consolidated financial statements. Iam 
pleased to report that my Auditor’s Report to the 
Legislative Assembly on the province’s consoli- 
dated financial statements for the year ended on 
March 31, 2009 is clear of any qualifications or 
reservations and reads as follows: 


To the Legislative Assembly of the Province 
of Ontario 


I have audited the consolidated statement of 
financial position of the Province of Ontario 
as at March 31, 2009 and the consolidated 
statements of operations, change in net 
debt, change in accumulated deficit, and 
cash flow for the year then ended. These 
financial statements are the responsibility of 
the Government of Ontario. My responsibil- 
ity is to express an opinion on these financial 
statements based on my audit. 
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I conducted my audit in accordance with 
Canadian generally accepted auditing stan- 
dards. Those standards require that I plan 
and perform an audit to obtain reasonable 
assurance whether the financial statements 
are free of material misstatement. An audit 
includes examining, on a test basis, evidence 
supporting the amounts and disclosures 

in the financial statements. The audit also 
includes assessing the accounting principles 
used and significant estimates made by the 
Government, as well as evaluating the over- 
all financial statement presentation. 


In my opinion, these consolidated financial 
statements present fairly, in all material 
respects, the financial position of the Prov- 
ince as at March 31, 2009 and the results 

of its operations, the change in its net debt, 
the change in its accumulated deficit, and its 
cash flows for the year then ended in accord- 
ance with Canadian generally accepted 
accounting principles. 


[signed] 


Jim McCarter, FCA 
Toronto, Ontario Auditor General 
July 31, 2009 Licensed Public Accountant 


Deficits and Debt 


EFFECT OF THE GLOBAL ECONOMIC 
DOWNTURN ON THE PROVINCE’S 
FINANCES 





The global economic downturn over the past year 
has severely impacted the province’s economy, 
causing tax revenues, especially from corporate 
taxes, to fall significantly. This was the main reason 
the province reported a deficit of $6.4 billion in its 
March 31, 2009 consolidated financial statements. 
The province’s total debt, which includes all prov- 
incial borrowing, has risen to $176.9 billion from 





$162.2 billion a year earlier. At the same time, 
Ontario’s net debt, which measures the extent 

to which the province’s total liabilities exceed 

its financial assets, is $153.3 billion, up from 
$142.4 billion a year earlier. Debt has increased 
more than this year’s deficit primarily because of 
the province’s investments in capital assets. 

In its 2009 Budget, the government projected 
a deficit of $14.1 billion by the end of 2009/10, 
$12.2 billion for 2010/11, and $9.7 billion for 
2011/12. The government said reduced revenues 
due to the global economic downturn and planned 
increases in government spending on provincial 
infrastructure and skills training over the next few 
years are primarily responsible for the projected 
deficits. Since the release of the 2009 Budget, 
Ontario, like other jurisdictions such as the federal 
government, has had to update its financial projec- 
tions. The Ministry of Finance (Ministry) in its 
Ontario Finances, First Quarter Update of June 30, 
2009, increased the 2009/10 fiscal-year deficit pro- 
jection to $18.5 billion from the $14.1 billion in the 
March budget. The Ministry attributed this higher 
deficit projection to a weaker-than-expected econ- 
omy and higher expenses arising from its support of 
the automotive sector. In its 2009 Ontario Economic 
Outlook and Fiscal Review, released on October 22, 
2009, the Ministry further increased its deficit 
projections to $24.7 billion for 2009/10, $21.1 bil- 
lion for 2010/11, and $19.4 billion for 2011/12. The 
Ministry attributes these higher deficits to further 
reductions in projected revenues due to a weaker 
economy than was previously anticipated and fur- 
ther increases in government spending. 

Figure 1 presents a summary of Ontario’s annual 
deficits and surpluses since the 1993/94 fiscal year 
when the province reported an $11.2 billion deficit, 
the highest ever recorded. 

The annual deficits and infrastructure spending 
that the government is projecting over the next 
few years will substantially increase the province’s 
debt. Although in its 2009 Budget the government 
projected its net debt would increase to $169.8 bil- 
lion by the end of the 2009/10 fiscal year, the 
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October 2009 Fiscal Review raised that projection to 
$184.1 billion, a 20% increase over the $153.3 bil- 
lion as of March 31, 2009. 


Like many governments around the world, 
Ontario continues to face significant economic 
challenges, and the government will need to 
establish a plan to return the province to a 
sustainable and firmer fiscal footing. In order 

to create jobs, help families, and position the 
province for future growth, the government 
has chosen to invest in infrastructure, in skills 
training, and in reshaping the tax system. As 
announced in the 2009 Ontario Economic Out- 
look and Fiscal Review, the government is tak- 
ing additional steps to ensure the sustainability 
of our public services. These initiatives will sup- 
port the continued relevance and effectiveness 
of government programs over the longer term. 
An update to the government’s plan to eliminate 
the deficit will be presented in the 2010 Budget. 


THE PROVINCE OF ONTARIO 
BORROWING PLAN 


The government must borrow in order to finance 
its deficits. It must also borrow to replace maturing 
debt and fund its investments in built or acquired 
capital assets. The borrowing outlook presented 
in the 2009 Ontario Budget projected that the 
government will need to borrow $34.8 billion in the 
2009/10 fiscal year. This projection was increased 
to $42.6 billion in the October 2009 Fiscal Review. 
The 2009/10 borrowing program reflected in 
the October 2009 Fiscal Review is significantly 
larger than last year’s $28.7 billion in borrowing 
and more than twice as large as in 2007/08. The 
2009/10 projected borrowing reflects the large 
deficits and planned increases in infrastructure 
spending over the next few years. 
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Figure 1: Provincial Surpluses/Deficits, 1993/94-2015/ 16 ($ billion) 


Sources of data: Province of Ontario Consolidated Financial Statements, 2009 Ontario Budget, and 2009 Ontario Economic Outlook and Fiscal Review 
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ing efficiencies to reduce the costs of running 
programs. However, given the size of the projected 
deficits reported in the October 2009 Fiscal Review 
of $24.7 billion, $21.1 billion, and $19.4 billion in 
2009/10, 2010/11, and 2011/12, respectively, the 
government will need to prepare an updated plan 
to eliminate the deficit. 


FINANCIAL CONDITION INDICATORS 


In 1997, the Canadian Institute of Chartered 
Accountants (CICA) published a research report, 
Indicators of Government Financial Condition, which 
suggested that a core set of indicators, common to 


all governments and presented on a consistent basis 
over time, would reduce the subjectivity of deter- 
mining whether a government’s financial condition 
is improving or deteriorating. 

In September 2008, the CICA’s Public Sector 
Accounting Board (PSAB) issued a draft statement 
of recommended practices. In March 2009, PSAB 
approved this draft and expects to release it in final 
form to the public in the near future. The draft 
recommends that a government adopt a specific 
framework when reporting on indicators of its finan- 
cial condition. It defines the government’s financial 
condition as its “financial health as assessed by its 
ability to meet its existing obligations both in respect 
of its service commitment to the public and financial 
commitment to creditors, employees, and others.” 

PSAB also suggests that governments calculate 
many of their specific financial indicators from 
information contained in their consolidated finan- 
cial statements, often in combination with public 
information about the economy. The province is 
currently reporting a number of these financial 
indicators in its Budget and Annual Report. 


ONTARIO FINANCIAL CONDITION 
IN BRIEF 


Ontario’s overall financial condition, as determined 
by PSAB’s recommended measures, will undoubt- 
edly deteriorate over the next few years, based on 
the government’s current projections. This is to be 
expected, given the recent slowdown in the provin- 
cial economy. 

It is important to note that the question of what 
the budgeted surplus or deficit should be or how 
much debt the government should incur is a policy 
decision outside the mandate of my Office. There- 
fore, our analysis of the province’s financial health 
is presented in order to demonstrate how this type 
of information can be useful to governments, legis- 
lators, and the public in assessing the province’s 
deteriorating financial condition. 

PSAB has recommended the use of several 
indicators to measure the sustainability, flexibility, 
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and vulnerability of government finances. Below, 
we concentrate on those indicators we consider 
most significant to help explain the impact of the 
large looming deficits and debt increases on the 
province’s financial condition. 


Sustainability 


Sustainability is the degree to which a government 
can maintain its existing financial obligations, 
both with respect to its service commitments to 
the public and financial commitments to creditors, 
employees, and others without increasing the debt 
or tax burden. Sustainability addresses the govern- 
ment’s ability to manage its financial and program 
commitments and debt burden. The cost of servi- 
cing increased debt levels can also have an impact 
on government programs as interest costs consume 
a greater proportion of revenues. Two key sustain- 
ability indicators are as follows. 


Net Debt as a Percentage of GDP 
Net debt, or liabilities minus financial assets, is a 
particularly significant measure of a government’s 
financial position as it reflects the future provincial 
revenues that will be required to dispose of its 
liabilities. Gross Domestic Product (GDP), the total 
value of all goods and services produced, is the 
critical measure of an economy’s output or wealth, 
which, over time, correlates to future expected 
provincial revenues. 

The net-debt-to-GDP ratio indicator measures 
the relationship between a government’s obliga- 
tions and its capacity to raise funds to meet them. 
When the ratio is rising, it means that the govern- 
ment’s net debt is growing at a faster rate than the 
provincial economy. 

The province’s net-debt-to-GDP ratio is expected 
to increase over the next three years, reflecting 
the government’s decision to significantly increase 
its borrowings in order to fund its deficits and 
infrastructure investments. However, as shown 
in Figure 3, the net-debt-to-GDP ratio has been 


Figure 3: Ratio of Net Debt to Gross Domestic Product 


(GDP), 1999/2000-2015/ 16 (%) 
Source of data: 2009 Ontario Budget 
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Note: Projections reflect amounts reported in the 2009 Ontario Budget but 
do not reflect the impact that the higher deficits reported in the 2009 Fiscal 
Review will have on this ratio because the information is not presented in the 
2009 Fiscal Review. 


falling steadily since the turn of the last decade. 

As reflected in the 2009 Ontario Budget, the ratio 
was projected to increase by more than one-third, 
from a low of 24.3% in the 2007/08 fiscal year to 

a high of 32.6% in 2011/12. However, given the 
revised deficit projections in the October 2009 Fiscal 
Review, the ratio is now expected to be higher and 
to increase to around 37% by 2011/12. 

The province’s March 31, 2009 annual report 
and consolidated financial statements use the ratio 
of the accumulated deficit to GDP as a key measure 
of Ontario’s fiscal economic health, consistent with 
the definition of provincial debt outlined in the 
Fiscal Transparency and Accountability Act, 2004. 
The net-debt-to-GDP ratio is also a key indicator of 
government sustainability because it deals with the 
debt that must be repaid relative to the value of the 
output of Ontario’s economy. 


Ratio of Net Debt to Total Annual Revenues 
The net-debt-to-total-annual-revenues ratio is an 
indicator of how much time would be needed to 
eliminate the province’s debt if all revenues could 
be devoted to it. As shown in Figure 4, the ratio 
of Ontario net debt to total annual revenues was 
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Figure 4: Net Debt as Percentage of Total Annual 


Revenues, 1999/2000-2011/12 
Source of data: 2009 Ontario Budget 
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Note: Projections reflect amounts reported in the 2009 Ontario Budget but 
do not reflect the impact that the higher deficits reported in the 2009 Fiscal 
Review will have on this ratio because the information is not presented in the 
2009 Fiscal Review. 





quite stable from the 1999/2000 fiscal year through 
2003/04, averaging 200% over the period. From 
2004/05 through 2007/08, there was a steady 
decline in the ratio to 147%, reflecting the fact that, 
while the province’s net debt remained essentially 
the same, annual provincial revenues were increas- 
ing. Although this positive trend was encouraging, 
we note from the 2009 Ontario Budget that from 
2008/09 onward, the ratio of net debt to total 
annual revenues was projected to increase each 
year and was expected to return to the 200% level 
by 2011/12. However, given the revised deficit 
projections in the October 2009 Fiscal Review, the 
ratio is now expected to be 238% by the end of the 
2011/12 fiscal year. 


Flexibility 


Flexibility measures the degree to which a govern- 
ment can change its debt or tax burden to meet 
existing financial obligations. Current borrowings 
reduce the government’s future ability to respond 
to adverse economic circumstances. Similarly, 
increasing taxes or government fees may reduce the 
government’s ability to levy such measures in the 





future as the government approaches the limits that 
the public is willing and able to bear. 
Again, we examine two indicators for this 


measurement. 


Interest Expenses as a Percentage of Provincial 

Revenues 
Increases in the cost of servicing total debt, or inter- 
est expenses, can directly affect the quantity and 
quality of programs and services that government 
can provide. The more government revenues that 
are needed to pay interest costs arising from past 
borrowing, the less will be available for program 
spending. 

The interest-expense-to-revenue ratio illustrates 
the extent to which servicing past borrowing takes 
a greater or lesser share of total revenues. 

As noted in Figure 5, the province’s interest- 
expense-to-total-revenues ratio has been decreasing 
steadily over the past decade, even as provincial 
net debt has been increasing. In fact, even with the 
significant increases in net debt over the next few 
years, the projected ratio is still much lower than 
what it was in the early 2000s due to an increasing 
revenue trend and lower projected interest rates. 


Figure 5: Interest Expense as Percentage of Total 


Revenues, 1999/2000-2011/12 
Source of data: 2009 Ontario Budget 
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Note: Projections reflect amounts reported in the 2009 Ontario Budget but 
do not reflect the impact that the higher deficits reported in the 2009 Fiscal 
Review will have on this ratio because the information is not presented in the 
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Based on the projected revenue and expense details 
provided in the 2009 Budget, and subsequently 
updated in the October 2009 Fiscal Review, the 
lower ratio is projected to continue through to 

the 2011/12 fiscal year, although it is projected to 
increase from the low in 2007/08. If this indicator 
increases for an extended period of time because of 
increased borrowing or higher interest rates, it will 
reduce the amount of flexibility the government 
has to spend money on programs providing public 
services. 

Interest rates have been relatively low and falling 
over the past several years, recently approaching 
record low levels. This has enabled the government 
to keep interest expenses relatively consistent even 
as its total borrowing has been increasing. 

The Risk Management and Derivative Financial 
Instruments note to the province’s March 31, 2009, 
consolidated financial statements states: “Based on 
floating rate interest-bearing financial instruments 
at hand at the balance sheet date plus planned 
refinancing of maturing debt in the coming year, a 
one-percent-increase in interest rates would result 
in an increase in interest expenses of $230 million.” 
In other words, if interest rates increase, even if 
debt levels remain constant, the government will 
pay more to carry this debt and thus have less flex- 
ibility to respond to future program needs. 


Own-source Revenue as a Percentage of GDP 
The ratio of own-source revenues, primarily tax and 
fee revenues, to GDP is important because it shows 
the extent to which a government is taking income 
out of the economy, either through taxation or user 
charges. If the indicator is increasing, it demon- 
strates that the government may have less room to 
raise taxes or increase fees. From the 2002/2003 
fiscal year to projections for 2011/12, the govern- 
ment’s own-source revenue as a percentage of GDP 
will likely hold steady, and, on the basis of projec- 
tions in the 2009 Ontario Budget, we estimate that 
it will range between 11.7% and 13.0% and average 
12.3%. The ratios are expected to remain in this 


range on the basis of the updated projections in the 
October 2009 Fiscal Review. 


Vulnerability 


Vulnerability refers to the degree to which a gov- 
ernment becomes dependent on outside revenue 
sources or is exposed to risks that could impair 

its ability to meet existing obligations, both with 
respect to its service commitments to the public and 
its financial commitments to creditors, employees, 
and others. It is an important aspect of financial 
condition because it provides insight into a govern- 
ment’s reliance on funding sources that are beyond 
its control and influence, such as revenue transfers 
from other levels of government. A government 
whose vulnerability is relatively low has greater 
control over its finances. 


Ratio of Federal Government Transfers to Total 

Revenues 
Although revenue projections have not been 
published beyond the 2011/12 fiscal year, the 
proportion of revenue that the Ontario govern- 
ment receives from the government of Canada has 
become more significant over time. Although 11.7% 
of the province’s revenue came from the govern- 
ment of Canada in the 2001/02 fiscal year, this per- 
centage has increased in each subsequent year and 
is projected to reach a high of 20.1% in 2009/10, 
falling back somewhat to 19.6% in 2011/12, as 
has been reported in the 2009 Ontario Budget. In 
reviewing the revised projection in the October 
2009 Fiscal Review, we note that the proportion 
of revenue that the Ontario government receives 
from the federal government follows the same pat- 
tern as projected in the 2009 Ontario Budget, with 
the ratio standing at 21.2% in 2009/10, 23.1% in 
2010/11, and 20.5% in 2011/12. This illustrates 
the dependence of the province on the government 
of Canada and that over the last decade the prov- 
ince’s vulnerability to Ottawa’s fiscal decisions has 
increased. The federal government is facing fiscal 
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problems of its own, and any future reductions in 
federal transfers could result in the province having 
to issue more debt or raise taxes or fees if it wishes 
to maintain its spending plans. 


THE CHALLENGES AHEAD 


Our analysis indicates that the province’s financial 
condition has generally been improving since the 
2001/02 fiscal year. However, this trend will reverse 
over the next few years due to the large deficits 

and debt increases the government is projecting. 
Although the resulting financial indicators at the 
end of the 2011/12 fiscal year based on the 2009 
Budget are not significantly out of line with the 
same indicators of a decade ago, the latest govern- 
ment projections indicate that further deterioration 


in the province’s financial condition is anticipated. 


Unspent Grants 





_ BACKGROUND 


Over the years I have expressed two concerns with 
respect to government year-end spending practices. 

The first related to the government flowing 
hundreds of millions, and even billions, of dollars 
at year-end to fund activities of future periods while 
expensing such grants as current-year expenses 
in its consolidated financial statements. This 
accounting presentation could convey that monies 
had been spent providing programs and services 
during that fiscal year, while in reality, little or no 
services had been provided and little or no benefits 
had been received by the public. However, I have 
acknowledged that accounting for such transfers 
as expenses was acceptable under Public Sector 
Accounting Board standards. 

My second concern focused on the weakening of 
normal accountability controls over transfers that 
occurred with these year-end grants, as the govern- 
ment, to meet accounting criteria allowing for their 


immediate expense recognition, often eliminated 
or significantly reduced the conditions under which 
such grants were provided to recipients. I was 
pleased to report last year that, after working with 
the Ministry, we were able to agree on a number 

of accountability and control provisions that could 
be included in these year-end transfers, addressing 
many of my concerns in this area. 


EXPENDITURES AND 2007/08 FISCAL 
YEAR-END TRANSFERS 


Year-end Grants 


In last year’s Annual Report, I also expressed 
concerns about the Investing in Ontario Act, 2008 
(Act, in this section). My primary concern was that, 
through this legislation, the government granted 
itself the power to determine how transactions 
would be accounted for in the province’s consoli- 
dated financial statements. My objection was that 
generally accepted accounting standards, not gov- 
ernment legislation, should determine how trans- 
actions are accounted for. I indicated that we would 
not support the use of legislative provisions to over- 
ride generally accepted accounting standards. 

In the 2007/08 fiscal year, the government 
provided an additional $1.1 billion in transfers to 
municipalities under the Act to fund infrastruc- 
ture investments. The proposed transfers were 
announced and committed to by the government 
before March 31, 2008, and, therefore, were prop- 
erly recorded as an expense in the 2007/08 fiscal 
year. The funds were actually transferred after 
the tabling of the 2007/08 Public Accounts in late 
August 2008. 

Municipalities generally had the discretion 
to spend these funds on their own infrastructure 
priorities. The transfers were subject to a number 
of accountability provisions, including: the require- 
ment to report by March 31 of each year, until the 
money was fully spent, how the funds were used; 
the right of the province to independently verify or 
audit the municipal expenditures; and the right of 
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the province to recover the funds if they were not 
used as intended. 

In addition to the $1.1 billion in transfers to 
municipalities under the Act, expenditures in the 
2007/08 fiscal year included $1.9 billion in year- 
end grants provided to a number of other transfer 
payment recipients. These included grants of 
$400 million to communities outside of Toronto for 
roads and bridges, $200 million to universities to 
maintain and upgrade their facilities, and $100 mil- 
lion in transfers for social housing infrastructure. 
Accountability provisions similar to those included 
with the transfers made under the Act were also 
incorporated into these grant awards. 


Unspent Grants 


In its 2009 Budget, the government indicated that 
it planned to make significant investments over the 
next two years to stimulate the provincial economy. 
This included some $32.5 billion in spending on 
infrastructure projects. The government noted this 
stimulus spending must not only be significant in 
size but must be effective in restoring growth. The 
government noted that, to be effective, the grants 
should support quick-start projects. In other words, 
the fund should not sit for extended periods in the 
bank accounts of transfer-payment recipients. 
Given the massive amount of stimulus spending 
that the government is planning to undertake over 
the next two years, the potential for some of these 
funds not to be spent cost-effectively is undoubt- 
edly a program risk. As well, economists and other 
academics have recently expressed concerns that 
despite governments’ stimulus spending commit- 
ments, many projects will be slow to start and that 
the actual investments may not be made for several 
years. In such cases, the “stimulus effects” will 
also not be felt for several years. As the following 
examples relating to last year’s year-end investments 
indicate, these concerns may have some merit. 
- During our audit of the 2008/09 Public 
Accounts, we noted that much of the $1.1 billion 


in infrastructure money provided to municipalities 
under the Act remained unspent. 

In accordance with PSAB standards, the 
amounts transferred to municipalities under the 
Act were recognized as a provincial expense for the 
2007/08 fiscal year. We reviewed funding provided 
to 25 of the 445 municipalities, representing over 
70% of the funds transferred under the Act, to 
determine the amounts that had been used as of 
March 31, 2009, approximately six months after the 
transfers flowed to the municipalities. Our observa- 
tions included: 

e Only two of 25 municipalities had fully spent 

100% of their funding, totalling $22 million. 
In fact, 15 municipalities had not spent any of 
the nearly $330 million they had received. 

e Seven municipalities had spent only a small 
portion of their grant monies: for example, 
one municipality spent only $141,000, or less 
than 0.2% of its $77.3 million; another spent 
only about $17,000, or less than 0.1% of its 
$20.2 million; and another spent $390,000, 
or 3.8% of the $10.3 million it had received. 
In total these seven municipalities spent only 
$10.8 million, or approximately 6%, of the 
total grants received of over $180 million. 

e@ One municipality was given permission to 
apply its $238 million grant to repay existing 
capital debt. 

At the time of our review, 390 of the 445 muni- 
cipalities had reported on the amount of funds 
used as of March 31, 2009. On the basis of these 
reports—and after removing the $238 million in 
debt repayments made by the one municipality— 
only $56 million of the remaining $910 million 
transferred to municipalities under the Act, or 6.2% 
of the total, had been spent. The Ministry explained 
that many municipalities had not used their fund- 
ing by March 31, 2009, because of construction- 
timing issues. As well, a number of municipalities 
were waiting to see if the funds provided under the 
Act could be used as the municipal portion for any 
federal stimulus funds available. 
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To summarize, although $1.1 billion was 
expensed in the 2007/08 fiscal year, only a fraction 
of the infrastructure investments have as yet been 
made. 

With respect to the $1.9 billion in other year-end 
grants, we reviewed seven of the more significant 
transfers, totalling $1.4 billion or approximately 
75% of the total, and noted that approximately 
$315 million, or only 22%, was spent by the end of 
the 2008/09 fiscal year. 


Enhancing Accountability 


Public accountability for any major year-end 
transfers and future stimulus funding that is to be 
spent over a multi-year period would be enhanced 
if the government publicly reported on the status 
of the money that it provided. Such reporting could 
be presented in the province’s annual report to 
clearly indicate the extent to which the funds trans- 
ferred have actually been spent on infrastructure 
investments. 


Since 2003, investing in infrastructure has been 
a priority for the government. Infrastructure 
investments are being made in all key sectors— 
including highways, roads, transit, environ- 
ment, health, education, culture, tourism, 
sport, recreation, and social housing projects. 
As part of the stimulus initiatives announced 

in the 2009 Budget, Ontario is supporting not 
only longer-term infrastructure investment 

but quick-start projects to help ensure that 

the stimulus investments are most effective. 

Of the $32.5 billion in planned infrastructure 
spending over the next two years, $6.9 bil- 

lion represents short-term federal—provincial 
stimulus spending. This stimulus funding was 
distributed using an application-based process 
for which a key selection criterion was appli- 
cants’ attestation of their ability to complete 
projects by the March 31, 2011 deadline. To 


date, the federal and provincial governments 
have already allocated about 75% of the stimu- 
lus funding to recipients, and as indicated in 
the government’s 2009 Economic Outlook and 
Fiscal Review of October 22, more than 650 
new projects are already under construction. 
Many municipalities were awaiting federal and 
provincial infrastructure stimulus allocations 
before directing the money provided to them 
under the Investing in Ontario Act, 2008 to pro- 
jects to optimize the value achieved for dollars 
spent. As announced in the 2009 Ontario Eco- 
nomic Outlook and Fiscal Review, the Ministry 
of Energy and Infrastructure will be launching 
a Revitalizing Ontario’s Infrastructure website 
that will allow Ontarians to see a list of infra- 
structure projects in their community and allow 
them to track the progress of these projects. 
This initiative is part of the government’s 
efforts to further enhance transparency and 
accountability in reporting to the public. 


Significant Accounting 


Issues 





The Public Sector Accounting Board sets out the 
objectives of government financial statements. The 
first and most fundamental of its standards is that: 
“Financial statements should provide an accounting 
of the full nature and extent of the financial affairs 
and resources which the government controls, 
including those related to the activities of its agen- 
cies and enterprises.” 

To meet this objective, the Ontario government’s 
annual financial statements currently include, 
through a consolidation process, the financial 
position and operating results of more than 300 of 
its most significant controlled agencies and enter- 
prises. These include its seven government business 
enterprises, including Ontario Power Generation, 
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the Ontario Lottery and Gaming Corporation, and 
the Liquor Control Board of Ontario; 47 other gov- 
ernment organizations, such as the Ontario Electri- 
city Financial Corporation, Ontario Place, and the 
Royal Ontario Museum; and, in the broader public 
sector, each of Ontario’s 153 public hospitals, its 
103 school boards or local school authorities, and 
its 24 colleges. 

However, there are currently five provincial enti- 
ties that are considered trusts and are therefore not 
included in the reporting of the province’s financial 
results. As such, the assets, liabilities, revenues, and 
expenses of these entities do not form part of the 
government’s consolidated financial statements. 
However, concerns regarding the deteriorating 
financial condition of two of these entities—the 
Workplace Safety and Insurance Board and the 
Pension Benefit Guarantee Fund—may warrant 
reconsideration of their exclusion. 


WORKPLACE SAFETY AND INSURANCE 
BOARD 


This year, we undertook a review of the Workplace 
Safety and Insurance Board’s (WSIB’s) unfunded 
liability. The result of our review is found in Chap- 
ter 3 of this Annual Report. Our review provides 
information on the significant recent growth in the 
unfunded liability, the factors contributing to this 
growth, and WSIB initiatives to control it. As of 
December 31, 2008, the WSIB’s unfunded liability 
was $11.5 billion—an increase of $3.4 billion, or 
42%, from the unfunded liability of $8.1 billion 

a year earlier. The WSIB is projecting that its 
unfunded liability will continue to grow over the 
next few years. 

Generally, trusts are excluded from the prov- 
ince’s financial results because the province has no 
access to the assets of the trust and has no respon- 
sibility to pay for any liabilities of the trust. How- 
ever, given the WSIB’s current unfunded liability, 
we believe a re-examination of the WSIB’s exclusion 
from the government reporting entity is warranted. 
Inclusion of the WSIB in the government’s financial 


reporting would have a material impact on its finan- 
cial statements. As of March 31, 2009, Ontario’s 
Statement of Financial Position reported provincial 
liabilities totalling $196.6 billion. These liabilities 
were offset by government assets of some $83.4 bil- 
lion, leaving a reported accumulated provincial 
deficit of $113.2 billion. As of December 31, 2008, 
the WSIB had liabilities of $24.7 billion, offset by 
assets of $13.2 billion, leaving it in a net unfunded 
liability of $11.5 billion. Accordingly, if the WSIB’s 
operations were to be included in the province’s 
financial statements, Ontario’s reported accumu- 
lated deficit would be increased by more than 10%. 

Inclusion of the WSIB would have had an even 
more dramatic effect on the province’s Statement 
of Operations, which reports the government’s total 
revenues and expenses for the year, thus provid- 
ing an accounting of the province’s annual deficit 
or surplus. The government reported a surplus of 
$600 million for the 2007/08 fiscal year and a defi- 
cit of $6.4 billion for 2008/09. However, the WSIB, 
whose results are excluded from the province’s 
expected results, lost $2.1 billion in 2007 and a 
further $3.4 billion in 2008. Accordingly, inclusion 
of the WSIB would have had a significant impact on 
the government’s reported results. 


Government Control of the WSIB 


The Public Sector Accounting Board sets out the 
principles and criteria governments and auditors 
should use in determining which organizations’ 
financial affairs and resources are to be included 
in the government’s financial statements. The key 
criterion for inclusion is that “the government 
reporting entity should comprise the organizations 
that are controlled by the government.” 
Accordingly, the first question to consider is 
whether the Ontario government controls the WSIB. 
PSAB defines control as the “power to govern 
the financial and operating policies of another 
organization.” This power need not be exercised, 
yet “control exists by virtue of the government’s 
ability to do so.” Whether this control exists is a 
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question of fact to be determined by the particular 
circumstances of each case, requiring an assess- 
ment of the substance of the relationship between 
the government and the organization, along with 
the application of professional judgment. PSAB 
outlines a number of control indicators that should 
be considered, adding that “it is the preponderance 
of evidence that would be considered in assessing 
whether a government controls an organization.” 
These control indicators are divided into two 
categories: ones that provide “more persuasive 
evidence of control” and others that “may provide 
evidence of control.” Figure 6 summarizes our 
assessment of the WSIB against both sets of indica- 


- tors. Our assessment makes extensive reference to 


provisions in the Workplace Safety and Insurance 
Act, 1997 (Act, in this section), the WSIB’s gov- 
erning legislation. 

Based on the “preponderance of evidence” pre- 
sented in Figure 6, the government meets the PSAB 
indicators that indicate it does “control” the WSIB. 
However, PSAB does provide one exception to its 
general rule that all controlled entities be included 
in the government reporting entity—“Trusts admin- 
istered by a government or government organiza- 
tion should be excluded from the government 
reporting entity.” 


Is the WSIB a Trust? 


The government of Ontario has classified the WSIB 
as a trust under this exception. Accordingly, the 
second question to consider is whether the WSIB is, 
indeed, a trust. Although PSAB standards provide 
some guidance to help assess whether this trust 
classification is appropriate for specific government 
organizations, the guidance is quite limited. It 
defines a trust as: “Property that has been conveyed 
or assigned to a trustee to be administered as 
directed by agreement or statute. In a trust relation- 
ship, the trustee holds title to the property for the 
benefit of, and stands in a fiduciary relationship to, 
the beneficiary.” 
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Figure 6: Assessment of Whether, under PSAB Definitions, the Government Controls the Workplace Safety and 


Insurance Board 
Prepared by the Office of the Auditor General of Ontario 
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Board Control Indicator 





The government has the power to unilaterally 
appoint or remove a majority of the members of 
the governing body of the organization. 


The government has ongoing access to the 
assets of the organization, has the ability to 
direct the ongoing use of those assets, or has 
ongoing responsibility for losses. 





THT ESS Cy 


The government appoints all WSIB members, including its president, as 
per section 162(1) of the Act. 


The government does not have ongoing access to the assets of the WSIB. 
Rather, most of the WSIB’s assets are in an insurance fund that provides 
benefits to injured workers. 


However, under the Act, the government has the ability to direct the 
WSIB’s assets’ ongoing use to fulfill the purposes of the Act. The most 
specific section of the Act in this regard is section 167. 


@ Section 167(1) of the Act states: 
“The Minister may issue policy directions that have been approved by 
the Lieutenant Governor in Council on matters relating to the Board’s 
exercise of its powers and performance of its duties under this Act.” 


@ Section 167(2) then requires the Board to adhere to these policy 


directions, as follows: 
“In exercising a power or performing a duty under this Act, the Board 
shall respect any policy direction that relates to its exercise.” 


The Act makes it clear that the WSIB is to be entirely funded by employer 
premiums. To date, the WSIB has never received any funds from the 
province. 


Section 96(4) of the Act states: 

“If the Lieutenant Governor in Council is of the opinion that the insurance 
fund is not sufficient to meet the standards described in subsections 

(2) and (3), the Lieutenant Governor in Council may direct the Board to 
increase employers’ premiums to the extent that the Lieutenant Governor 
in Council considers necessary to ensure that the funds meet those 
standards.” 


However, the Act also makes clear that the government is the ultimate 
backstop for the WSIB’s financial obligations, as section 100, in part, 
states: 

“The following rules apply if there is not sufficient money available in the 
insurance fund to make the required payments as they become due... the 
Lieutenant Governor in Council may direct that an amount be advanced to 
the Board from the Consolidated Revenue Fund to make the payments.” 


Given the significant recent unfunded liability, we believe it is no longer 
certain that the province’s “backstop” will not be required. The province 
also appeared to recognize this as, in referring to responsibility for the 
unfunded liability of trusts, the 2007 financial statements stated, “they will 
be discharged by external parties,” while the 2008 statements noted that 
“it is intended they be discharged by external parties.” 


As noted in our review of the WSIB’s unfunded liability in Chapter 3, the 
government does influence the setting of employer premium rates and the 
level of benefits to be paid. As such, the actions of the government can 
have a significant impact on the annual operating results of the WSIB and 
its unfunded liability position. 
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Assessment 


indicator met 


indicator met 
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Board Control Indicator Discussion Assessment 
The government holds the majority of the As indicated above, the government appoints the WSIB’s chair, its indicator met 
voting shares that confers the power to govern _ president, and all other board members. Also, as indicated above, section 

the financial and operating policies of the 167(1) of the Act gives the Minister broad powers to direct the WSIB, 

organization. and the WSIB is obligated under the Act to adhere to such orders. The 


combination of these two powers is equivalent to the “golden share” 
concept, which the Public Sector Accounting Board defines as, “powers 
or rights generally exceeding those normally associated with the holder's 
ownership interest or representation on the governing body.” 





The government has the unilateral power to The original Worker's Compensation Board (WCB) and the more recent indicator met 
dissolve the organization and thereby access WSIB were both created by the provincial government through legislation. 

its assets and become responsible for its Unlike some of the other entities the government consolidates, such as 

obligations. many of Ontario's public hospitals, the WSIB was not established by a 


party outside the government, nor has it any outside ownership or equity 
interests. Given that the WSIB was created solely by the government of 
Ontario and is accountable only to it, and although highly unlikely, the 
government has the power to dissolve it, solely at its discretion. The 
control criteria clearly states that the government merely has to have the 
power and does not require any assessment of whether this power would 
be exercised. 


The government has the power to provide As indicated above, under section 162.1, the government appoints all indicator met 
significant input into the appointment of members of the WSIB’s governing body. 

members of the governing body of the 

organization by appointing a majority of those 

members from a list of nominees provided 

by others or being otherwise involved in the 

appointment or removal of a significant number 

of members. 


The government has the power to appoint or Under section 162(1)(b), the government appoints the WSIB’s president. —_ indicator met 
remove the CEO or other key personnel. 


The government has the power to establish The government has the power to amend the WSIB’s underlying legislation. indicator met 
or amend the mission or mandate of the For instance, in 1997, the Workers’ Compensation Act was significantly 
organization. revised, including revisions to the board’s mission and mandate, as well 
as its name. 
The government has the power to approve the —_No section of the Act speaks directly to the government’s power to indicator met 
business plans or budgets for the organization | approve the WSIB’s business plans or budgets. However, section 16/, 
and require amendments, either on a net or as indicated above, requires the WSIB to adhere to any policy direction 
line-by-line basis. issued by the government. This section provides the government with 


sufficient power to direct any aspect of the WSIB’s operations, including 
the specifics of its business strategy or its spending plans. 


The government has the power to establish Under section 97(4), the WSIB’s investments are restricted to those indicator met 
borrowing or investment limits or restrict the authorized under the Pension Benefits Act, another provincial statute. 


wed lly UME oe Section 166 requires that the WSIB and the Minister enter into a 


memorandum of understanding and requires that the memorandum 
state that “the Board must give the Minister an annual statement of its 
investment policies and goals.” 


The government also has the authority to restrict the WSIB’s investments 
further at any time by virtue of section 167, which allows it to issue policy 
direction to the Board. 
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Public Sector Accounting 
Board Control Indicator 


The government has the power to establish 
or amend the policies that the organization 
uses to manage, such as those relating 

to accounting, personnel, compensation, 
collective bargaining, or deployment of 
resources. 


Discussion Assessment 


Historically, the government has left it up to the WSIB to establish its indicator met 
internal administration policies. However, the Act does provide the 


government with the power to influence these policies should it so desire. 
As noted earlier, section 167(1) states: 


“The Minister may issue policy directions that have been approved by the 
Lieutenant Governor in Council on matters relating to the Board’s exercise 
of its powers and performance of its duties under this Act.” 


This section is sufficiently broad to cover any of WSIB’s management 
policies, such as those to relating to accounting practices or employee 
compensation. 


The memorandum of understanding requirements under section 166 are 
also another legislative mechanism by which the government can (and 
does) establish or amend WSIB’s management policies. 


With respect to administrative matters, section 131(1), in part, states: 
“...With the approval of the Lieutenant Governor in Council, the Board may 


make rules governing its practice and procedure.” 





After this somewhat legalistic definition, PSAB’s 
commentary on trusts is limited to a few subse- 
quent paragraphs. It says: 


To meet the definition, the property con- 
veyed or assigned to the government or 
government organization acting as trustee 
must be provided to fulfill a particular 
objective of the donor of the property 
conveyed or assigned. The government or 
government organization would merely 
administer the terms and conditions 
embodied in the agreement and has no 
unilateral authority to change the condi- 
tions set out in the trust indenture. 


Furthermore, PSAB also says: “Often the term 
‘trusts’ is applied to assets earmarked as a result of 
a government policy decision when no trust liability 
exists. Such assets are special funds that comprise 
part of the government reporting entity and would 
be consolidated.” 

To supplement this guidance, we reviewed the 
characteristics of both private- and public-sector 
trusts and assessed their resemblance to the WSIB. 
The WSIB does not operate like a trust. One feature 
of almost all trusts is the absence of significant busi- 
ness risk. Property is almost invariably conveyed to 


trusts for limited time periods and for eventual dis- 
position under specified conditions to one or more 
beneficiaries known to and specified by the donor. 
The trust manages the conveyed property until the 
conditions for transfer specified by the donor arise. 
Accordingly, for every asset of significance held by 
a trust, there is a comparable offsetting liability to 
a specified beneficiary. Given this offsetting nature 
of trust assets and liabilities, exclusion of trusts 
administered by a government from its financial 
statements makes sense, as inclusion would 
unnecessarily clutter the statements with extran- 
eous “noise” on both sides of the balance sheet that 
would be of little use in assessing the government’s 
own financial position or condition. 

The Office of the Public Guardian and Trustee is 
arguably the best example of a trust administered 
by the province. This government office manages 
the assets of financially incapable individuals and 
the estates of persons who die intestate without 
adult heirs. It also manages funds for minors until 
they reach the age of majority and administers the 
property of dissolved corporations. Finally, it acts 
as the depository for all monies, mortgages, and 
securities paid into the Superior Court of Justice 
awaiting court disposition. As of March 31, 2009, 
this office held total assets and offsetting liabilities 


of some $1.3 billion. These were appropriately 
excluded from the province’s consolidated state- 
ment of financial position. 

Another key PSAB requirement states that to 
be a trust the property conveyed must “fulfill a 
particular objective of the donor,” and that the 
government as trustee must have “no unilateral 
authority to change the conditions set out in the 
trust indenture.” In fact, the Ontario government 
has unilateral authority, which it exercises periodic- 
ally, to legislate changes to, among other things, the 
benefits paid to the WSIB’s “beneficiaries.” 

Particularly given the recent significant increase 
in the size of the unfunded liability, we are becom- 
ing increasingly of the opinion that the government 
is running an insurance business through the 
WSIB rather than administering a trust. If so, the 
government is subject to the same business risks as 
the owners of any insurance company. Like insur- 
ance companies, the WSIB charges and collects 
premiums that it establishes based on its actuarial 
projections of expected future claims. These 
projections take into account both past claims 
experience and management’s beliefs about the 
future. These premiums are not then maintained 
for eventual return to a beneficiary specified by 
the donor. Rather, as with insurance organiza- 
tions, premiums are pooled in insurance funds and 
invested. Significant business risk arises from this 
operation, because the claims that may eventually 
arise and have to be paid from insurance funds may 
far exceed the value of premiums collected and the 
investment returns these premiums were able to 
generate. That the WSIB is subject to considerable 
business risk is obvious from the very fact that it 
has now accumulated an unfunded obligation that 
is $11.5 billion and growing. 

The bottom line is that if the WSIB was operat- 
ing like a true trust, it would not have such a signifi- 
cant unfunded liability. 

There is a final point to be made regarding this 
trust classification issue. Even if one accepts the 
classification of the WSIB as a trust, it is worth 
examining the accounting standards for defined 
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benefit pension plans, another type of trust admin- 
istered by most governments. Unlike trusts such 

as the Office of the Public Guardian and Trustee, 
sponsors of defined benefit pension plans are also 
subject to significant business risk. This is because 
defined benefit pension plans ultimately provide 
their beneficiaries with pensions that are not 
directly and necessarily related to or derived from 
the specific contributions received on their behalf. 
Pensions payable from such defined benefit plans 
are typically based on a formula driven by such 
factors as the employee’s average salary during the 
years just preceding his or her retirement and the 
number of years of employment service. The pen- 
sion obligation arising from the application of this 
formula may differ significantly in value from the 
value of contributions received on the employee’s 
behalf and the investment returns the plan has been 
able to earn on these contributions over the course 
of the employee’s career. 

To reflect these business risks appropriately, 
while continuing to respect the fact that the assets 
in pension plans are held in trust, accounting 
standards for defined benefit pension plans require 
that the plan sponsors include on their statement of 
financial position not all of the plan’s assets and lia- 
bilities, but rather only the net surplus or shortfall 
in the pension plans they sponsor. A pension asset 
on the plan sponsor’s books reflects the fact that 
the assets are currently in excess of its obligations, 
while a pension liability reflects the fact that its 
liabilities are currently in excess of its assets. 

The government of Ontario applies these stan- 
dards in accounting for each of the defined benefit 
pension plans it sponsors, the three largest of which 
are the Public Service Pension Plan, the Ontario 
Public Service Employees Union Pension Plan, and 
the Ontario Teachers’ Pension Plan. As of March 31, 
2009, the province in its consolidated financial 
statements disclosed that its pension benefit liabil- 
ities, primarily for these three plans, amounted to 
$68.1 billion. The province further disclosed that 
these liabilities were offset by plan fund assets, 
unamortized actuarial gains, and other adjustments 


of $72.9 billion, resulting in a net asset position for 
the government of $4.8 billion. This $4.8 billion 
net asset was appropriately included along with 

all the other government assets on its statement of 
financial position. 


Government Accountability 


No other jurisdiction in Canada has a workers’ 
compensation board with such a large unfunded 
liability. It is often said that what gets measured 
gets more attention than what does not. Inclusion 
of the WSIB in the government reporting entity 
would mean that its annual surplus or deficit would 
form part of the province’s overall annual surplus 
or deficit, the prime measure used by the media, 
the public, and the government itself of its overall 
fiscal performance. It would also mean that the 
WSIB’s unfunded liability would form part of the 
government’s overall accumulated deficit, a key 
measure of its net financial position. Exclusion may 
convey the message that the WSIB’s financial chal- 
lenges are not the government’s problem. Exclusion 
may also provide less of an incentive for the govern- 
ment to help address the WSIB’s financial problems. 
Consequently, and notwithstanding the technical 
accounting arguments for inclusion or exclusion, 
inclusion of the WSIB in the province’s consolidated 
financial statements would enhance government 
accountability and transparency. 

In conclusion, we believe that, particularly in 
light of the recent significant increase in the WSIB’s 
unfunded liability, the government should formally 
re-evaluate its current policy of excluding the WSIB 
from the province’s financial statements. 


PENSION BENEFITS GUARANTEE FUND 


The Pension Benefits Guarantee Fund (PBGF), 
established in 1980 under the Pension Benefits Act 
(Act, in this section), is the only fund of its kind 
in Canada. Its purpose is to act as the guarantor 
of last resort for certain pension benefits when 
eligible defined benefit plans are wound up under 
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conditions specified in the Act. The Superintendent 
of the Financial Services Commission of Ontario is 
responsible for the administration of the fund. 

The PBGF is funded through premiums charged 
to and paid by private-sector pension plan sponsors. 
Similar to the WSIB, the PBGF is classified as a trust 
in the province’s consolidated financial statements 
because PBGF liabilities are not considered to be 
obligations of the province. As such, its assets, 
liabilities, and operating results are excluded from 
the government reporting entity but are disclosed 
in the notes to the province’s consolidated financial 
statements. 


2008/09 Fiscal Year Financial Condition 


Similar to our concerns with respect to the WSIB, 
we believe recent events necessitate a review of the 
PBGF’s exclusion from the financial statements. 

As the result of claims made over the last few 
years, the PBGF has an unfunded liability of over 
$47 million as of March 31, 2009. This means that 
bona fide claims exceed the assets available to pay 
them. This liability position is in spite of the PBGF 
receiving a $330 million non-interest-bearing loan 
from the province in the 2003/04 fiscal year. The 
government has discounted this loan to its effective 
value of $162 million to reflect its non-interest- 
bearing nature and its repayment arrangements of 
$11 million in annual instalments over a 30-year 
period. Since the $330 million loan was made, the 
province has provided the PBGF with an additional 
loan of $30 million. 

The PBGF’s financial health continues to be of 
concern because a number of companies spon- 
soring pension plans are in significant financial 
distress and could make claims which, according 
to the notes to the PBGF’s March 31, 2009, finan- 
cial statements, “could significantly exceed [its] 
existing assets.” These potential claims would in 
all likelihood put the fund in the position where 
it would be unable to meet its claim obligations 
or repay its provincial loans. The government 
has partially recognized this risk by increasing its 


provision for uncollectible loans in the province’s 
books. This reflects the increasing risk that the 
PBGF could default on its scheduled future loan 
repayments. 

The current poor financial condition of the PBGF 
raises the further risk that its remaining liabilities 
will become the responsibility of the province. The 
continued direct provincial assistance to the PBGF 
implies that its trust classification may no longer 
meet generally accepted accounting standards for 
treatment as a trust. 

The government is fully aware of the PBGF’s 
financial difficulties. Its Expert Commission on 
Pensions in November 2008 recommended that 
an examination be conducted to determine the 
appropriate fees and guarantees needed to ensure 
the PBGF is governed on self-financing principles. 
The commission also recommended that the PBGF 
be administered at arm’s length from the pension 
regulator. 

In response to these recommendations, the 
government has appointed an independent actuary 
to review the stability and the financial status of the 
PBGF. Once the study is completed, the government 
is expected to consider establishing an independent 
PBGF agency. Its objective would be to ensure that 
the PBGF operates on sound business principles 
with coverage and assessment levels that are sus- 
tainable over the long term. | 

The government has since amended the Act 
to clarify that the PBGF is a self-sustaining fund 
independent of the government. The amendments 
include provisions allowing, but not requiring, the 
government to provide grants or loans to the PBGF. 
The amended Act also emphasizes that the PBGF’s 
liabilities are limited by its assets. 

Similar to our concern about the significant 
deteriorating financial condition of the WSIB and 
whether it should continue to be excluded from the 
province’s financial statements, we recommend the 
government formally re-evaluate the continued 
exclusion of the PBGF from the province’s financial 
statements. 
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The Ministry, in conjunction with the Office of 
the Auditor General, will review the province’s 
current accounting treatment for the Workplace 
Safety and Insurance Board and the Pension 
Benefits Guarantee Fund, taking into considera- 
tion the current economic circumstances. 


Status of Public Accounts 


Issues Raised in Prior Years 





INTRODUCTION 


As noted last year, from time to time my Office and 
the Ministry of Finance may have differing views 
on the most appropriate accounting treatment of 
certain issues. This is not uncommon between the 
preparers and the auditors of an entity’s financial 
statements, and typically we work together to 
resolve our differences. As a result, my predeces- 
sor and I have been able to issue an unreserved or 
“clean” opinion on the annual consolidated finan- 
cial statements of the government since the prov- 
ince first adopted Public Sector Accounting Board 
standards in the 1993/94 fiscal year—a period of 
16 years. In my view, this demonstrates the com- 
mitment of the government to prepare consolidated 
statements in accordance with generally accepted 
accounting principles for governments. 

During this year’s audit of the government’s 
consolidated financial statements, we again dealt 
with a number of accounting issues. Of particular 
note were two issues raised in prior years where 
we had a difference of opinion: accounting for 
certain transfers to the province by other levels of 
government for investments in provincial capital 
infrastructure, and accounting for rate-regulated 
assets and liabilities. Because of the size of the gov- 
ernment of Ontario and the dollar value of its trans- 
actions, these issues were not significant enough 


to affect my opinion on the overall fairness of the 
province’s consolidated financial statements. How- 
ever, we indicated we would continue to work with 
the Ministry in an attempt to resolve these matters 
in the 2008/09 fiscal year. Both of these issues are 
discussed below. 


ACCOUNTING FOR CAPITAL TRANSFERS 


We noted in last year’s Annual Report that the gov- 
ernment was not accounting for all capital transfers 
it received from other levels of government in 
accordance with PSAB standards. Under these stan- 
dards, capital transfers should be recognized as rev- 
enues when the province incurs the expenditures 
that make it eligible to receive the grant. We noted 
the province had received significant federal grants, 
accumulated over several years, that in our view 
should have been recognized as revenue because 
the government had incurred the expenditures 
making it eligible for the grants in question. How- 
ever, the recognition of these revenues has been 
deferred, reflecting the government’s view that 
these revenues should be recognized over the use- 
ful lives of the related assets. As of March 31, 2009, 
these deferred amounts continued to grow but are 
not yet significant enough to impact the fairness of 
the consolidated financial statements. 

We are awaiting the new standards regarding 
accounting for capital transfers because the PSAB 
standards on government transfers are currently 
being revised. 


RATE-REGULATED ASSETS AND 
LIABILITIES 


Rate regulation refers to an arrangement whereby 
a government-established authority approves 

the prices that a regulated entity can charge its 
customers for its products or services. Regulators 
often prohibit regulated entities from immediately 
recovering all of their current costs in their current 
rates, ordering rather that such costs be “deferred” 
(and recorded as an asset) for recovery in future 
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periods. Rate-regulated accounting practices were 
developed to recognize the unique nature of regu- 
lated entities and these types of transactions. 

In last year’s Annual Report, we discussed 
our concerns regarding the use of rate-regulated 
accounting in the government’s consolidated finan- 
cial statements. The discussion focused on a num- 
ber of our technical arguments against the practice. 
Our concerns remain unresolved. However, rather 
than repeat our previous arguments, this year we 
would like to approach the issue at a more concep- 
tual level. 

A key purpose of government consolidated 
financial statements is to provide an objective, 
consistent, comparable, and sound measure of 
the government’s surplus or deficit for the fiscal 
period and of its accumulated deficit at the end of 
the fiscal period. Arguably, the essential principle 
that must be applied to ensure that credible results 
are arrived at is the use of a consistent appropriate 
criterion for revenue and expense recognition. This 
is often referred to as the “basis of accounting.” 
PSAB calls for government financial statements to 
be prepared using the accrual basis of accounting. 
Under this approach, revenues are recognized in 
the periods they are earned, and expenses are rec- 
ognized in the periods they are incurred. 

We fully support the accrual basis of accounting 
and believe that, with one exception, it is being 
applied consistently in the government and its 
consolidated entities. The exception is the govern- 
ment’s electricity sector. 

The government owns and controls all of 
the major entities in Ontario’s electricity sector, 
including Ontario Power Generation Inc. (OPG), 
Hydro One Inc., the Ontario Electricity Financing 
Corporation, the Ontario Energy Board, and the 
Ontario Power Authority (OPA). Each of these 
entities was created by the government to deliver 
on particular aspects of its overall energy policies. 
The basis of accounting used by three of these enti- 
ties—OPG, Hydro One, and OPA—is rate-regulated 
accounting. The financial results of each of these 
three entities are combined with those of all other 


significant government-controlled organizations 
to arrive at the government’s consolidated finan- 
cial position and results of operation. However, 
although expenses are recorded when incurred for, 
all other consolidated entities such as ministries, 
large Crown agencies, school boards, colleges, and 
hospitals, electricity entities can recognize certain 
expenses as assets through the regulatory process. 

Although we acknowledge that the use of rate- 
regulated accounting may be acceptable in the 
entities’ individual financial statements, we believe 
that using the same basis of accounting as the 
province for determining its consolidated results 
would be more conceptually sound. The rationale 
for allowing certain expenses to be deferred and 
recorded as assets is that the expenses will be 
recovered from future revenues from electricity 
consumers. In essence, the anticipated revenues are 
being recorded as assets in advance in the account- 
ing of the province. 

PSAB standards contain no allowances for the 
advance recognition of any other types of future 
revenues such as revenues from taxes, liquor sales 
or profits, casino revenues or profits, government 
fees or fines, or any other government revenues 
that may be as assured or even more assured than 
future electricity-sector profits. They also contain 
no allowances for the deferral of any current 
government costs to future periods, regardless of 
the degree of certainty that such costs can or will 
be recovered. In fact, the government’s annual 
reported deficit or surplus would have little mean- 
ing if such cost deferrals and future revenues 
recognition were allowed under PSAB accounting 
standards. Yet this is what rate-regulated account- 
ing allows for in the electricity sector. And the 
numbers are significant—the two biggest electri- 
city-sector entities recognized $877 million in rate- 
regulated assets and $661 million in rate-regulated 
liabilities between them as of their December 31, 
2008, year end. 

We believe this issue needs to be specifically 
addressed by PSAB—not from the perspective 
of the regulated entity but from the perspective 
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of the government’s own consolidated financial 
statements. 

To précis some of the technical arguments 
We made in last year’s Annual Report, we do not 
believe that rate-regulated assets and liabilities 
meet the definition of bona fide assets or liabilities 
under generally accepted accounting principles 
for governments. We also believe that, from the 
perspective of the government as opposed to that 
of the regulated entity, there is no independence 
of the regulator from the organization being 
regulated and from the government itself. Both 
the regulator and the regulated entity are owned 
and controlled by the government that created 
them. Without such independence, one could argue 
that the government itself is deciding what costs 
do not need to be recognized as expenses rather 
than applying established accounting principles in 
making that determination. From this perspective, 
assets and liabilities arising from rate-regulated 
gains and losses should be removed as part of the 
consolidation adjustment process just like all other 
inter-organizational gains and losses to arrive at a 
fair presentation of the government’s transactions 
with external parties. 

We noted last year that the Canadian Institute of 
Chartered Accountants was adopting international 
accounting standards as part of its move to har- 
monize Canada’s accounting practices with those 
found around the world. At that time, these new 
international standards did not contain provisions 
supporting rate-regulated accounting, and the 
CICA’s Accounting Standards Board had indicated 
that it did not intend to amend these standards 
with regard to such accounting practices. Rather, it 
indicated that all assets and liabilities would have 
to meet the CICA’s conceptual framework defin- 
itions to be included in financial statements. We 
were encouraged by these developments because 
rate-regulated assets and liabilities do not meet 
these definitions. 

However, this year, the International Account- 
ing Standards Board issued an exposure draft that, 
if approved, would allow rate-regulated entities 





to continue recognizing regulatory assets and 
liabilities under certain conditions. This proposed 
accounting standard will in all likelihood be used by 
the province’s electricity-sector businesses. Accord- 
ingly, our hope that our concerns with respect to 
rate-regulated accounting would be addressed 
through the adoption of international standards no 
longer appears as likely. This makes it all the more 
important for PSAB to address the issue directly 
and to do so from the perspective of the govern- 
ment’s consolidated financial statements. For now, 
we acknowledge that we have little choice under 
existing standards but to allow the continued use 
of rate-regulated accounting in accounting for 
government business enterprises in the province’s 
consolidated financial statements, even though we 
question its conceptual basis. 


The Government Reporting | 


Entity 





FULL LINE-BY-LINE CONSOLIDATION OF 
THE BROADER PUBLIC SECTOR 


Under the Public Sector Accounting Board’s 
reporting entity standard, governments have been 
permitted to consolidate broader-public-sector 
(BPS) organizations on a modified equity basis of 
accounting up to and including the 2008/09 fiscal 
year. Under modified equity accounting, BPS net 
assets have been reported as a single line item on 
the province’s Consolidated Statement of Finan- 
cial Position, and each sector’s net expenses are 
included as a single line on the province’s Schedule 
of Expenses. Currently, there are three BPS sec- 
tors that are affected in Ontario: hospitals, school 
boards, and colleges. 

For all fiscal years that commence on or after 
April 1, 2009, the PSAB standard requires BPS 
organizations to be fully consolidated. Full consoli- 
dation means that the accounts of BPS organiza- 
tions are to be included using the same accounting 
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policies as the province, with each revenue and 
expense item, as well as each asset and liability 
item, being combined with the corresponding item 
in the province’s consolidated financial statements. 
One key consequence of this line-by-line approach 
is that the $33.9 billion in BPS tangible capital 
assets and $15.5 billion in BPS net debt from these 
three sectors would form part of the province’s total 
capital assets and net debt, respectively. 

In January 2009, PSAB extended the transition 
period for the consolidation of these organizations 
on a line-by-line basis by one year. The government 
views a one-line consolidation for these sectors as 
best representing the bottom-line fiscal accountabi- 
ilty of these organizations to the province for man- 
aging these public funds. 

We reviewed the 2008/09 financial statements 
of the other provinces to see how other jurisdictions 
are dealing with this standard and note that the 
majority of Canadian jurisdictions have already 
adopted full line-by-line consolidation of their BPS. 

We are currently working with the Ministry of 
Finance on the presentation of these BPS organ- 
izations in the province’s consolidated financial 
statements and of the impact on the consolidated 
financial statements if full line-by-line consolida- 
tion is not adopted. 


Capital Asset Accounting and 


Reporting 





In January 2003, PSAB revised a 1997 standard set- 
ting out the rules for the recognition, measurement, 
amortization, and presentation of capital assets in 

a government’s financial statements. The standard 
recommends that governments, in a manner similar 
to the approach taken in the private sector, record 
acquired or constructed capital items as assets and 
amortize their cost to operations over their esti- 
mated useful lives. 


The government’s approach has been to phase in 
the adoption of these recommendations over time. 
In its 2002/03 fiscal year, the government valued 
and capitalized the province’s land holdings, build- 
ings, and transportation infrastructure, and for 
the first time recognized over $13 billion of its net 
capital investments in these assets. As of March 31, 
2009, the province’s net investments in these 
capital assets had grown to $21.7 billion, and these 
assets are now appropriately recorded on the prov- 
ince’s consolidated statement of financial position. 

The government has advised us that it is com- 
pleting the capitalization project for its remaining 
tangible capital assets, including computer systems, 
vehicles, aircraft, and marine fleet, in the 2009/10 
fiscal year. We have met with ministry officials a 
number of times to address the scope of this project 
and the method of valuing and accounting for these 
assets. 


Public Sector Accounting 


Board Initiatives 





This section briefly outlines some of the more 
significant issues that the Public Sector Accounting 
Board has been dealing with over the last year that 
may in future affect the province’s consolidated 
financial statements. 


INTRODUCTION 


The Canadian Institute of Chartered Accountants’ 
Public Sector Accounting Standards Board (PSAB) 
has the authority to set accounting standards 

for the public sector. PSAB is working to address 

a number of complex financial accounting and 
reporting issues, including accounting for finan- 
cial instruments, government transfers, foreign 
exchange, and how the adoption of International 
Financial Reporting Standards by government 
business enterprises and public-sector not-for-profit 
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organizations should be accounted for in the gov- 
ernment’s consolidated financial statements. 

The Accounting Standards Board of the CICA, 
the national organization responsible for establish- 
ing Canadian accounting and financial reporting 
standards, is implementing a number of financial 
reporting changes to be used by all publicly traded 
companies. By 2011, the current Canadian gener- 
ally accepted accounting principles used to prepare 
the financial statements of publicly accountable, 
profit-oriented enterprises will be replaced by an 
accounting framework set out in International 
Financial Reporting Standards. The Accounting 
Standards Board is also reviewing and updating the 
standards applicable to not-for-profit organizations. 
These changes reflect the ongoing globalization 
of financial markets and the movement toward 
worldwide standards in several areas of business 
and government. 


STANDARDS 
Financial Instruments 


The province uses financial instruments and 
derivatives such as foreign-exchange forward 
contracts, swaps, futures, or options to manage 

or hedge against risks related to debt it has issued 
in foreign currencies and/or at variable interest 
rates. Currently, PSAB guidance on accounting for 
derivatives is limited to their application in hedg- 
ing foreign-currency items, such as managing the 
foreign-currency risk associated with holding a 
debt repayable in U.S. dollars. 

In January 2005, the CICA’s Accounting Stan- 
dards Board approved three new handbook sections 
relating to such activities: “Financial Instruments,” 
“Comprehensive Income,” and “Hedges.” Although 
these handbook sections were developed for the 
private sector, and governments were not required 
to apply them, they underscored the need to even- 
tually address these issues from a public-sector 
perspective. Accordingly, PSAB created a task force 
to consider how governments should account for 


financial instruments. One of the key issues it will 
address is whether changes in the fair market value 
of derivative contracts, similar to fluctuations in 
the market value of equities and bonds, should be 
recognized in a government’s financial statements. 
A key aspect of this issue is whether such changes 
should affect the determination of a government’s 
annual surplus or deficit. 

The main rationale for recognizing changes in 
the fair market value of financial instruments is to 
ensure that assets and liabilities of an organization 
are recognized at their current value rather than 
their historical acquisition value at the end of each 
fiscal period. However, if such changes in value 
were recognized as immediate gains or losses, they 
could have a significant impact on the organiza- 
tion’s annual surplus or deficit, even though such 
gains or losses may not have been realized and 
could be reversed in future years. 

PSAB issued its exposure draft on financial 
instruments in September 2009. Among its more 
significant recommendations is that all gains and 
losses from fair value re-measurement be recorded 
in the Statement of Operations and that these 
re-measurement gains and losses be reported 
separately from the province’s other revenues and 
expenses so that the province’s surplus or deficit 
clearly distinguishes the impact of re-measurement 
gains and losses. PSAB notes that the recom- 
mendations contained in it will bring the financial 
accounting and reporting of financial instruments, 
including derivatives, in line with international 
developments. These proposed standards are essen- 
tially consistent with the accounting used by the 
private sector. 


Foreign Currency Translation 


At present, PSAB standards include recommen- 
dations allowing gains and losses on foreign- 
currency-denominated items to be deferred and 
amortized to operations over time. PSAB notes that 
its accounting standard is the only one among the 
major accounting standards used throughout the 
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world that allows deferral and amortization of such 
foreign-exchange gains and losses, and that this 
accounting is not consistent with its conceptual 
framework or asset and liability definitions. 

PSAB has indicated that, as part of its plan to 
address financial instruments, it will need to revisit 
these recommendations. Specifically, it is expected 
that the current deferral provisions will be replaced 
with the requirement that such gains and losses 
be immediately recognized as re-measurement 
gains and losses in the determination of the annual 
surplus or deficit. PSAB has indicated that it intends 
to approve an exposure draft on foreign currency 
translation in the near future. We expect it to incor- 
porate the changes discussed above. 


Government Transfers 


PSAB has been working over a number of years 

to amend its standard on government transfers to 
address a number of issues raised by the govern- 
ment community. Although there are a number of 
issues that need to be addressed, the main issue to 
be resolved is how multi-year funding for capital 
transfers provided by one government to another 
should be accounted for. Given the billions of dol- 
lars in government transfers made annually, the 
revised standard has the potential to significantly 
affect a government’s reported financial results. 

A variety of views have been expressed and 
PSAB has faced challenges in obtaining a consensus 
on what revisions should be made to the existing 
standard. The more recent re-exposure draft, issued 
in April 2009, proposed that operating transfers be 
recognized as revenue in the period the transfer is 
authorized and any eligibility criteria is met, unless 
the transfer gives rise to a liability for a recipient 
government. The proposal is consistent with PSAB’s 
conceptual framework but allows for more profes- 
sional judgment in assessing whether a liability 
exists. PSAB is currently reviewing responses to this 
second re-exposure draft. 


Financial Reporting by Government Not- 
for-profit Organizations 


Currently, government not-for-profit organizations, 
such as hospitals, colleges, and universities, are 
directed by PSAB to follow the CICA standards for 
not-for-profit organizations. The Accounting Stan- 
dards Board is in the process of evaluating options 
for future financial reporting and accounting stan- 
dards for private-sector not-for-profit organizations. 
In June 2009, the Accounting Standards Board 

and PSAB issued a joint invitation to comment 

on financial reporting by not-for-profit organiza- 
tions. There are a number of financial reporting 
options available for these entities, and a key issue 
is whether all not-for-profit organizations should 
apply the same standards. Although this would 
enhance comparability, some believe not-for-profit 
organizations should have the flexibility to choose 
from alternative sets of standards based on their 
particular circumstances. 

The options for public-sector not-for-profit organ- 
izations under consideration include using PSAB 
standards alone or PSAB standards supplemented 
by certain CICA not-for-profit standards. Both 
the Accounting Standards Board and PSAB have 
rejected developing a set of stand-alone standards to 
accommodate all not-for-profit organizations. 

PSAB has indicated that it intends to issue an 
exposure draft on this subject in the near future. 


Liability for Remediation and Mitigation of 
Contaminated Sites 


Canadian accounting standards currently provide 
no guidance on accounting for environmental 
liabilities. In recognition of the unique complexities 
associated with such liabilities, PSAB approved an 
environmental liability project in June 2006. As 

the project progressed, it decided to limit the scope 
of the project to the development of a proposed 
accounting standard on the remediation and 
mitigation of contaminated sites. In January 2009, 
PSAB approved a statement of principles for this 


Public Accounts of the Province 


proposed standard and in April 2009 issued it for 
public comment. 

The statement of principles notes that only legal 
obligations should be recognized as liabilities. 
Obligations that may arise from intention or poli- 
cies that are not legally enforceable should not be 
recognized in the financial statements. PSAB has 
indicated that the next stage in the process will be 
to issue an exposure draft on liability for contamin- 
ated sites. 


Tax Revenue 


Given the importance and magnitude of tax 
revenue, PSAB approved an exposure draft in Nov- 
ember 2007 to address many of its unique issues. A 
re-exposure draft was released in April 2009 that 
took into account stakeholder responses to the first 
exposure draft. The re-exposure draft proposes an 
accounting standard that calls for the recognition 
of tax revenues when they meet the definition of an 
asset, are authorized, and the taxable event occurs. 
PSAB expects to approve a final standard on tax 
revenues later this year. 


Revenue from Exchange Transactions 


Revenue in the public sector is generated from both 
exchange and non-exchange transactions. PSAB is 
currently addressing certain revenues arising from 
non-exchange transactions, such as government 
grants received, in its government transfers project 
and its tax revenue project. Exchange transactions 
are not currently defined by PSAB. However, the 
International Accounting Standards Board (IASB) 
currently has a project on revenue recognition and 
has established a definition for exchange trans- 
actions as those “in which one entity receives assets 
or services, or has liabilities extinguished, and dir- 
ectly gives approximately equal value (primarily in 
the form of cash, goods, services, or use of assets) 
to another entity in exchange.” Examples of Ontario 
government exchange transactions include liquor 
and lottery ticket sales. 


Revenue recognition has a direct impact on the 
measurement of the surplus and deficit reported 
by governments. Although governments receive 
a significant portion of their revenue from non- 
exchange transactions, many governments still 
receive a substantial portion of their revenues from 
exchange transactions, as do government organ- 
izations. Accordingly, PSAB approved a project 
proposal on this topic in June 2009. 


GUIDANCE 
Assessment of Tangible Capital Assets 


In November 2008, PSAB released a statement of 
recommended practice to assist governments in 
reporting on their major assets and to improve the 
comparability and reliability of financial and non- 
financial information about such assets. It is also 
intended to assist governments in evaluating their 
financial condition and their financial and non- 
financial performance. 

Existing guidance on reporting financial and 
other information about tangible capital assets is 
limited. Appropriate information about the use and 
condition of a government’s tangible-capital-asset 
infrastructure assists users in understanding the 
ongoing maintenance, renewal, and replacement 
costs associated with this infrastructure. It is 
therefore a major factor in assessing a govern- 
ment’s financial ability to maintain existing levels of 
services. 


Statutory Matters 


Under section 12 of the Auditor General Act, Iam 





required to report on any Special Warrants and 
Treasury Board Orders issued during the year. In 
addition, section 91 of the Legislative Assembly 
Act requires that I report on any transfers of money 
between items within the same vote in the Esti- 
mates of the Office of the Legislative Assembly. 


ay 2009 Annual Report of the Office of the Auditor General of Ontario 


LEGISLATIVE APPROVAL OF 
EXPENDITURES 


Shortly after presenting its budget, the govern- 
ment tables detailed Expenditure Estimates in the 
Legislative Assembly outlining, on a program-by- 
program basis, each ministry’s spending proposals. 
The Standing Committee on Estimates (Committee) 
reviews selected ministry estimates and presents 

a report on them to the Legislature. The estimates 
of those ministries that are not selected for review 
are deemed to be passed by the Committee and are 
so reported to the Legislature. Orders for Concur- 
rence for each of the estimates reported on by the 
Committee are debated in the Legislature for a 
maximum of two hours and then voted on. 

Once the Orders for Concurrence are approved, 
the Legislature provides the government with legal 
spending authority by approving a Supply Act, 
which stipulates the amounts that can be spent by 
ministry programs, typically those set out in the 
estimates. Once the Supply Act is approved, the 
individual program expenditures are considered to 
be Voted Appropriations. The Supply Act pertaining 
to the fiscal year ended March 31, 2009, received 
Royal Assent on April 23, 2009. 

The Supply Act is typically not passed until well 
after the start of the fiscal year—and sometimes 
even after the related fiscal year—but ministry pro- 
grams require interim funding approval prior to its - 
passage. The Legislature authorizes these payments 
by means of motions for interim supply. For the 
fiscal year ending March 31, 2009, the Legislature 
passed a motion of interim supply on December 6, 
2007 that covered the period April 1, 2008 to 
July 31, 2008. The government also passed two 
acts allowing interim appropriations—the Interim 
Appropriation Act, 2008, and the Supplementary 
Interim Appropriation Act, 2008. These two acts 
received Royal Assent on May 14, 2008, and Nov- 
ember 27, 2008, respectively, and authorized the 
government to incur up to $87.5 billion in public 
service expenditures, $2.6 billion in investments 
in capital assets, and $195.9 million in legislative 


office expenditures. Both acts were made effective 
as of April 1, 2009, and provided the government 
with sufficient temporary appropriation authority 
to allow the government to incur expenditures 
from April 1, 2008 to March 31, 2009. The motion 
of interim supply provided the government with 
temporary approval to incur expenditures until 
the Interim Appropriation Act, 2008 received Royal 
Assent. - 

Because the legal spending authority under 
the Interim Appropriation Act, 2008 and the 
Supplementary Interim Appropriation Act, 2008 was 
intended to be temporary, the acts were repealed 
under the Supply Act, 2009, and the authority to 
incur expenditures provided under the acts were 
subsumed into the authority provided under the 
Supply Act, 2009. 


SPECIAL WARRANTS 


If motions for interim supply cannot be approved 
because, for instance, the Legislature is not in ses- 
sion, section 7(1) of the Treasury Board Act, 1991 
allows for the issuance of Special Warrants author- 
izing the incurring of expenditures for which there 
is no appropriation by the Legislature or for which 
the appropriation is insufficient. Special Warrants 
are authorized by Orders-in-Council approved by 
the Lieutenant Governor on the recommendation of 
the government. 

There were no Special Warrants issued for the 
fiscal year ended March 31, 2009. 


TREASURY BOARD ORDERS 


Section 8(1) of the Treasury Board Act, 1991 allows 
the Treasury Board to make an order authorizing 
expenditures to supplement the amount of any 
voted appropriation that is expected to be insuffi- 
cient to carry out the purpose for which it was 
made. The order may be made only if the amount of 
the increase is offset by a corresponding reduction 
of expenditures to be incurred from other voted 
appropriations not fully spent in the fiscal year. The 


Public Accounts of the Province Cm 


order may be made at any time before the books of 
the government of Ontario for the fiscal year are 
closed. The government considers the books to be 
closed when any final adjustments arising from our 
audit have been made and the Public Accounts have 
been tabled in the Legislature. 

Subsection 5(4) of the Treasury Board Act, 
1991 allows the Treasury Board to delegate to 
any member of the Executive Council or to any 
public servant employed under the Public Service 
of Ontario Act, 2006 any power, duty, or function 
of the board, subject to limitations and require- 
ments that the board may specify. For the fiscal 
year ended March 31, 2009, the Treasury Board 
delegated its authority for issuing Treasury Board 
Orders to ministers for making transfers between 
programs within their ministries and to the Chair of 
the Treasury Board for transfers between programs 





in different ministries and making supplementary 
appropriations from contingency funds. Supple- 
mentary appropriations are Treasury Board orders 
whereby the amount of an appropriation is offset by 
reducing the amount available under the govern- 
ment’s centrally controlled contingency fund. 

Figure 7 summarizes the total value of Treasury 
Board Orders issued for the past five fiscal years. 
Figure 8 summarizes Treasury Board Orders for the 
fiscal year ended March 31, 2009, by month of issue. 

According to the Standing Orders of the Legis- 
lative Assembly, Treasury Board Orders are to 


Figure 7: Total Value of Treasury Board Orders Issued, 
2004/05-2008/09 ($ million) 
Source of data: Treasury Board 
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Figure 8: Treasury Board Orders by Month of Issue, 
2008/09 


Source of data: Treasury Board 


Month of Issue # 





Authorized ($) 











April 2008-February 2009 67 _2,436,569,000 
March 2009 22 (67 4293982900. 
April 2009 18 _65,003,200 


September 2009 55,800. 


be printed in The Ontario Gazette, together with 


explanatory information. Orders issued for the 
2008/09 fiscal year are expected to be published in 
The Ontario Gazette in December 2009. A detailed 
listing of 2008/09 Treasury Board Orders, showing 
the amounts authorized and expended, is included 
as Exhibit 3 of this report. 


TRANSFERS AUTHORIZED BY THE 
BOARD OF INTERNAL ECONOMY 


- When the Board of Internal Economy authorizes 
the transfer of money from one item of the esti- 
mates of the Office of the Assembly to another item 
within the same vote, section 91 of the Legislative 
Assembly Act requires that we make special mention 
of the transfer(s) in this Annual Report. 

Accordingly, with respect to the 2008/09 Esti- 
mates, the following transfers were made within 
Vote 201 and Vote 202, respectively: 








From: lItem2 Office of the Clerk $ 20,700 
To: Item 3 Legislative Services $ 20,700 
From: Item 10 Members’ Office Support Services $132,200 
To: Item 8 Caucus Support Services $ 132,200 
From: Item 2 Office of the Information and 

Privacy Commissioner $ 30,400 
To: Item 1. Environmental Commissioner $ 30,400 
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UNCOLLECTIBLE ACCOUNTS 


Under section 5 of the Financial Administration Act, 
the Lieutenant Governor in Council, on the recom- 
mendation of the Minister of Finance, may author- 
ize an Order-in-Council to delete from the accounts 
any amounts due to the Crown that are deemed 
uncollectible. The amounts deleted from the 
accounts during any fiscal year are to be reported in 
the Public Accounts. 

In the 2008/09 fiscal year, receivables of 
$390.2 million due the Crown from individuals and 
non-government organizations were written off (in 
2007/08, the comparable amount was $199.6 mil- 
lion). The major portion of the write-offs related to 
the following: 

@ $138 million for uncollectible corporate tax 

(2007/08 — $59.8 million); 
@ $126.5 million for uncollectible retail sales tax 
(2007/08 — $92.4 million); 
@ $40.9 million for uncollectible motor 
fuel, tobacco, and land transfer tax 
(2007/08 — $0.5 million); 

@ $25.9 million for uncollectible employer 

health tax (2007/08 — $10.4 million); 

e $15.7 million for uncollectible Criminal Code 

fines (2007/08 — $7.3 million); 
® $14.9 million for uncollectible receivables 
under the Student Support Program 
(2007/08 — $9.9 million); and 

@ $12 million for uncollectible receivables 
under the Ontario Disability Support Program 
(2007/08 — $5.6 million). 

Volume 2 of the 2008/09 Public Accounts 
summarizes the write-offs by ministry. Under the 
accounting policies followed in the preparation of 
the consolidated financial statements of the prov- 
ince, a provision for doubtful accounts is recorded 
against accounts receivable balances. Accordingly, 
most of these write-offs had already been expensed 
in the government’s consolidated financial state- 
ments. However, the actual deletion from the 
accounts required Order-in Council approval. 





Our value-for-money (VFM) audits are intended to 
examine how well government, organizations in 
the broader public sector, agencies of the Crown, 
and Crown-controlled corporations manage 

their programs and activities. These audits are 
conducted under subsection 12(2) of the Auditor 
General Act, which requires that the Office report 
on any cases observed where money was spent 
without due regard for economy and efficiency or 
where appropriate procedures were not in place to 
measure and report on the effectiveness of service 
delivery. This chapter contains the conclusions, 
observations, and recommendations for the value- 
for-money audits conducted in the past audit year, 
except for those previously published in a special 
report during the year. 

The ministry programs and activities and the 
organizations in the broader public sector audited 
this year were selected by the Office’s senior man- 
agement on the basis of various criteria, such as a 
program’s or organization’s financial impact, its sig- 
nificance to the Legislative Assembly, related issues 
of public sensitivity and safety, and the results of 
past audits and related follow-up work. 

We plan, perform, and report on our value-for- 
money work in accordance with the professional 
standards for assurance engagements, encompass- 


Reports on 
Value-for-money Audits 


ing value for money and compliance, established by 
the Canadian Institute of Chartered Accountants. 
Accordingly, our audits include such tests and other 
procedures as we consider necessary in the circum- 
stances, including obtaining advice from external 
experts when needed. Our testing generally focuses 
on activities and transactions conducted in the most 
recently completed fiscal year. 

Before beginning an audit, our staff conduct 
in-depth research into the area to be audited and 
meet with auditee representatives to discuss the 
focus of the audit. During the audit, staff maintain 
an ongoing dialogue with the auditee to review the 
progress of the audit and ensure open lines of com- 
munication. At the conclusion of the audit field- 
work, which is normally completed by late spring of 
that audit year, a draft report is prepared, reviewed 
internally, and then discussed with the auditee. 
Senior Office staff meet with senior management 
from the auditee to discuss the draft report and to 
finalize the management responses to our recom- 
mendations. In the case of organizations in the 
broader public sector, discussions are also held with 
senior management of the funding ministry. All 
responses are then incorporated into the report in 
each of the VFM sections. 
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Section 


Program 


Background 


The Assistive Devices Program (Program) is admin- 
istered by the Ministry of Health and Long-Term 
Care (Ministry). The primary objective of the Pro- 
gram is to provide support and funding to Ontario 
residents with long-term physical disabilities to 
obtain personalized assistive devices that enable 
them to function more independently. 

Each category of device is funded differently. In 
general, devices can only be purchased from vend- 
ors who are registered with the Program. In most 
cases the client pays a portion of the equipment’s 
cost at the time of purchase, and the vendor from 
whom he or she purchases it bills the Ministry for 
the balance. The exceptions are supplies for which 
the client receives a grant from the Program and 
may purchase supplies from any vendor he or she 
wishes. 

A client’s first access to the Program is often 
through a diagnosing physician. Another health- 
care professional who is registered with the Pro- 
gram as an “authorizer” then assesses the client’s 
needs and prescribes the appropriate devices or 
supplies. The client then selects a vendor that sells 
him or her the prescribed device or supplies. 

Figure 1 shows the 2008/09 fiscal year expendi- 
tures spread across various device categories for a 


Chapter 3 Ministry of Health and Long-Term Care 


mike Assistive Devices 


Figure 1: Assistive Devices Program Expenditures by 


Device Categories, 2008/09 ($ million) 
Source of data: Ministry of Health and Long-Term Care 


entereal feeding ($4) eR 


communications 
and visual aids ($9) 


ostomy grant ($12) 
orthotics ($14) 


mobility aids 
($105) 









prosthetics ($17) 


insulin pumps and 
supplies ($18) 


respiratory devices 
($33) 


home oxygen 
($71) 


hearing aids ($60) 
total of $347 million. Program expenditures have 
increased by more than 90% over the $181 million 
spent in 2001/02, the time of our last audit. This 
increase can be attributed to a price adjustment 
in 2004 to reflect fair market prices, as Program- 
approved prices had not been adjusted since 1993, 
an increase in the number of program clients from 
173,000 to 294,000, and the introduction of a new 
insulin pump and supplies program in 2006. 





Audit Objective and Scope 





The objective of our audit was to assess whether the 
Ministry has effective systems and procedures to: 

e ensure that program payments and resources 
are managed economically and efficiently, and 
in accordance with eligibility and other policy 
requirements; and 

@ measure and report on its achievement of 
program performance and objectives. 

We developed audit criteria to assess the 
adequacy of the key systems, policies, and pro- 
cedures that should be in place and operating 
effectively. Senior ministry management reviewed 
and agreed to these criteria. We then designed and 
conducted tests and procedures for meeting our 
audit objective and criteria. 

To conduct our audit, we reviewed relevant 
ministry files, policies, and procedures. We inter- 
viewed appropriate ministry staff, reviewed sup- 
porting documents from vendors and health-care 
professionals, obtained relevant information from 
stakeholder groups and from comparable programs 
in other jurisdictions, and used computer-assisted 
audit techniques to analyze claims data. The work 
of the Ministry’s Internal Audit Services did not 
affect the extent of our work because it had not 
recently conducted any audits of the Program. 


Since our last audit in 2001, the Ministry of Health 
and Long-Term Care’s Assistive Devices Program 
has improved its ability to monitor and enhance 
service delivery to clients. However, we believe that 
the Program can be run more cost effectively if the 
Ministry manages program payments more eco- 
nomically and enforces eligibility and other policy 
requirements more rigorously. 


Assistive Devices Program A 88: 


Specifically, the Ministry should more frequently 
review the prices it pays for goods and services 
and the prices and fees that vendors charge the 
Program’s clients to ensure that they are reason- 
able. Because many of the clients who rely on this 
Program have to pay a portion of the cost of their 
devices, they are also adversely affected when the 
Ministry sets or accepts prices that are significantly 
higher than fair market value. The Ministry also 
needs to increase its efforts to identify and address 
the risks and costs related to ineligible claims, 
unusual claim patterns, and overpayments. Finally, 
the Ministry should be more proactive in identify- 
ing and addressing potential conflict of interest 
between authorizers and vendors and in pursuing 
other potentially questionable practices. 

With respect to enhancing and monitoring servi- 

ces to clients: 

e@ The Ministry has implemented several good 
initiatives to improve customer service. It has 
standardized claims-processing and response 
times, prioritized the assignment of work, and 
put procedures in place to investigate com- 
plaints and maintain complaint records. 

e Asa means of monitoring service-delivery 
levels, the Ministry conducts Customer 
Satisfaction Surveys every two years, and has 
re-instated standing committees to provide 
advice on policy, eligibility criteria, and the 
development of program-evaluation and mon- 
itoring strategies. 

With respect to ensuring that competitive prices 

are being paid for assistive devices: 

e@ In 2004, the Management Board of Cabinet 
granted the Ministry an exemption from 
competitive tendering for home oxygen after 
accepting the Ministry’s proposal to negotiate 
a contractual agreement with representatives 
of home oxygen vendors. In its approval, 
the Management Board stated that annual 
expenditures are “not to exceed $54.6 million 
annually’. We found the Ministry expenditures 
exceeded the approved amount by $6 mil- 
lion to $11 million for each of the fiscal years 
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from 2004/05 to 2007/08, and, although the 
Management Board approved a reallocation 
of funds from other program areas within the 
Ministry to fund this, there was no documen- 
tation to indicate that this issue had been spe- 
cifically addressed. We also found, that from 
the 2002/03 to the 2008/09 fiscal years, the 
Ministry paid a total of $2.2 million more than 
the amount set in the existing agreement with 
vendors delivering home oxygen to clients in 
northern areas. The Ministry told us that the 
agreement was inconsistent with the intent 

of program policy and that it would seek to 
amend the agreement. 

The oxygen concentrators that are supplied 

to clients by vendors cost between $400 and 
$1,000 and last five to seven years. Based on 
the monthly fee ($389) that vendors receive 
for providing home oxygen, the Ministry pays 
them about $23,000 for each client over a 
five-year period. Although that Ministry indi- 
cated that a significant portion of the $23,000 
related to other service-related costs incurred 
by the vendors, such as staffing and adminis- 
tration, the Ministry had not formally analyzed 
the reasonableness of this nor compared the 
price to that being paid by other provinces. 
We noted from our test sample that vendor 
mark-ups in all major device categories were 
higher than the reasonable target of 33% set 
by the Ministry. Average mark-ups for mobility 
devices, respiratory devices, and computer 
systems were 84%, 117%, and 128%, respect- 
ively, because the Ministry reviews and sets 
the Program-approved prices for these devices 
every two years without full consideration of 
significant price decreases in the marketplace 
arising from recent technological advances 
for certain types of devices. The prices set by 
the Ministry also do not take into account the 
potential for some vendors to obtain volume 
discounts. 

The Ministry allowed computer components 
such as monitors, printers, and scanners an 


even higher mark-up, which enables vendors 
to bill computer equipment to the Program 

at significantly higher than market prices. 

For example, the Program-approved price is 
$1,332 for a monitor that often costs vendors 
only about $250, resulting in a potential 
mark-up of 400%. In our testing of the rea- 
sonableness of prices of computer systems 
with monitors and printers, we obtained price 
quotes from five Program-registered vend- 

ors. The prices quoted ranged from $1,300 

to $4,400. The vendor that quoted $4,400 
offered to cover the client’s portion of $1,100 if 
the purchase was eligible for program funding. 


With respect to the monitoring of claims: 
e The Ministry reviewed scooter claims in 


2004/05. Its review resulted in the termina- 
tion of the agreement with an authorizer who 
had authorized scooters for individuals who 
were not eligible for program funding. The 
ministry review had a deterrence effect in the 
year immediately following, as evidenced by a 
13% drop in total scooter claims, but the effect 
was short-lived, as indicated by an increase 

in scooter claims of 109% from 2005/06 to 
2008/09. We reviewed three vendors (two of 
whom were also reviewed by the Ministry in 
2004/05) whose scooter claims had increased 
by more than 800% over the last three years, 
going from $88,000 to $805,000. Our review 
indicated that the Ministry was not consist- 
ently monitoring scooter claims to identify 
unusual claim patterns and take appropriate 
action to prevent potential abuses. 

Certain other provinces use independent res- 
piratory therapists to assess clients’ continued 
eligibility for home oxygen, but Ontario uses 
respiratory therapists employed by oxygen 
vendors to perform such assessments. The 
obvious risk associated with vendor-employed 
respiratory therapists assessing clients’ eligi- 
bility is that it is in the vendor’s interest for the 
client to continue to receive home oxygen. 


e Respiratory therapists employed by home 


oxygen vendors perform annual assessments 
of home oxygen clients to support their con- 
tinued need for home oxygen, but they are not 
required to submit the results to the Ministry 
unless requested. One-third of the sample of 
client assessments we requested from vend- 
ors had either not been done or had results 
indicating that the clients no longer met the 


Assistive Devices Program 55 


e@ Even in cases where the Ministry did find 


potential conflict of interest or misconduct on 
the part of Program-registered health-care 
professionals, it seldom took action to termin- 
ate their agreements with the Program and 
alert the regulatory college or professional 
association. In some cases, the Ministry knew 
about a problem for several years yet took no 
remedial action. 


criteria for long-term home oxygen supply. Yet With respect to recycling and refurbishing 
the Ministry continued to pay for these clients wheelchairs for reuse: 


to receive home oxygen. e The Ministry has contracted with a vendor 


Claims for Frequency Modulated (FM) 
Systems, a type of hearing device that 
minimizes background noise to make the 
speech signal more pronounced, have risen 
significantly since 2004/05, especially in the 
senior age group (66 and over), whose claims 
increased by almost 1,800% from 187 claims 
or $250,000 in 2004/05 to 3,557 claims or 
$4.8 million in 2008/09. Some clients indi- 
cated that their FM systems came in “pack- 
ages” with hearing aids and they did not really 
need or use the FM systems. The Ministry 
developed a plan of action in January 2009 
to identify improper claims and investigate 
irregularities, to prevent further abuses. 


to exclusively provide clients throughout 
Ontario with both new and recycled power 
wheelchairs from March 2007 to February 
2010. The vendor guaranteed a recycling rate 
of 20% in its first year of operation and 25% 
thereafter, with any shortfall to be credited 
to the Ministry, but we found that the actual 
recycling rate in the first year was 8.4%, and 
the rate for the second year has yet to be 
determined. After we brought this issue to 
ministry staffs attention, they advised us that 
they would follow up with the vendor. 

Since 2002/03, manual wheelchairs have 
accounted for about 80% of the Program’s 
wheelchair claims. However, the Ministry 
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With respect to detecting and deterring poten- currently has no recycling initiative in place 


tial conflict of interest between authorizers and for used manual wheelchairs. We found that 
vendors: other jurisdictions such as Alberta and Quebec 


e The Ministry should be more proactive and have programs in place to recycle and refur- 


rigorous in detecting and deterring potential 
conflicts of interest among vendors, author- 
izers, and/or prescribers in all major device 
categories. We found that some vendors had 
more than 90% of their claims signed by only 


bish manual wheelchairs for reuse. Aside from 
the environmental impact, these provinces 
were able to achieve significant cost savings of 
$4 million to $5 million per year, because the 
average cost of a recycled wheelchair was only 


one or two authorizers or prescribers. One about one-third of a new one. 


such vendor had claimed more than $10 mil- With respect to recovering overpayments: 
lion for hearing aids since 2000. We also e The Ministry has identified payments that 
found that some authorizers or prescribers were made to vendors as far back as 2001 
had been continually referring clients to the for deceased clients whose home oxygen 
same vendors, located more than 30 kilo- payments continued to be made after their 
metres away, although many other Program- death. The Ministry was already attempting to 


registered vendors were located much closer recover these funds. 


to where the clients lived. 
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e We identified potential duplicate payments 
for clients’ claims made by the Ministry 
and the Workplace Safety and Insurance 
Board (WSIB). Since 2006, the Ministry has 


recovered about $110,000 in duplicate funding 


for hearing aids, but it was not aware of and 
had not recovered duplicate funding for other 
device categories until we brought this to the 
Ministry's attention. Ministry staff indicated 
that this was because there is no information- 
sharing agreement in place with the WSIB. 





The Ministry is dedicated to the fair and 
responsible delivery of the Assistive Devices 


Program to ensure that program recipients, who 
are among Ontario’s most vulnerable citizens, 
have access to the assistive devices and supplies 
that they require. The Program provides fund- 
ing support to enable these clients to obtain 
competitively priced, personalized assistive 
devices appropriate for the individual’s basic 
needs. Increasing utilization of the program is 
the result of Ontario’s aging population, and the 
increased independence of seniors and people 
with long-term physical disabilities who are able 
to continue to live in their own communities 
instead of living in more costly institutional 
settings. The Ministry generally accepts the rec- 
ommendations of the Auditor General and will 
continue its efforts to strengthen accountability 
and to ensure the efficient use of resources and 
the provision of high quality devices at reason- 
able prices. 

The Ministry initiated work in the 2008/09 
fiscal year to improve the transparency of the 
procurement of Home Oxygen services by mov- 
ing to a Vendor of Record list, strengthening 
the registration requirements for home oxygen 
vendors, and clarifying the requirements for 
long-term oxygen therapy eligibility. As part of 
this work, the Ministry is reviewing the pricing 
structure for the provision of home oxygen. It is 


continuing to increase its efforts in compliance 
and quality assurance and is implementing 
new procedures and across-the-board training 
in risk management. The number of confirma- 
tion letters sent to approved clients has been 
increased by 193% since the 2003/04 fiscal 
year, and a contact management system is being 
implemented to improve stakeholder relations. 
The amount and quality of information avail- 
able to the public through the Ministry’s website 
is being increased to improve transparency on 
device-listing, availability of vendors, and eligi- 
bility criteria. Beginning in the 2008/09 fiscal 
year, the Ministry has been working to imple- 
ment a new information system to replace the 
current legacy system by spring 2011, which will 
help the Ministry to monitor patterns and trends 
of authorizer and vendor activity to ensure that 
program payments are managed in accordance 
with the Program’s policy requirements. 





Detailed Audit Observations 


OVERVIEW OF MAJOR DEVICE 
CATEGORIES AND KEY PLAYERS 


Each category of assistive device is funded dif- 
ferently and involves different players. Figure 2 
provides an overview of how funding works for 
each major device category. Figure 3 defines and 
illustrates the key players (authorizers and vend- 
ors) involved in the program. 


PROGRAM PERFORMANCE 
Client Service Delivery 


Since our last audit in 2001, the Ministry has 
improved its ability to enhance and monitor its 


service delivery to clients. Some of the initiatives the 


Ministry has undertaken include: 
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Figure 3: Assistive Devices Program - Key Players 
Source of information: Ministry of Health and Long-Term Care 











Key Players _ Description General Registration Requirements Roles and Responsibilities 





registered 
authorizer 


Se eee 


° private business or non-profit 


registered 
vendor 


* qualified health-care 





* sign and submit Authorizer 
Agreement, which stipulates 

Program terms and conditions of retaining 

about 6,000 authorizers in authorizer status with Program 

various professions listed * provide proof of professional 

with Program: physicians, qualifications and good standing 

audiologists, occupational with professional college or 

therapists, physiotherapists, association 

speech pathologists, 

optometrists, and 

ophthalmologists 


works in hospitals, home- 
Care agencies, and practices 
assessing Clients’ needs 

and prescribing appropriate 
devices or supplies 

in some device categories 
authorizer can also be vendor, 
eg., hearing aids 


professional registered with 


* sign and submit the Vendor 
Agreement, which stipulates the 
terms and conditions of retaining 
vendor status with Program 


organization registered with . 
Program 


supply assistive devices or 
supplies to persons eligible * complete an application and 

for program funding provide various business 

about 1,000 vendors listed documents, including proof of 
with Program; some sell ownership, insurance and banking 


products in more than one information, manufacturers’ 
device category agreements, proof of staff's 


professional qualifications, and floor 


plan/office layout 

° if vendor works out of multiple 
locations, each must be registered 
separately with Program 








* meet all conditions specified in the 
Authorizer Agreement 

* authorize equipment that fits client's 
functional requirements and meets 
program eligibility criteria 

* inform client about program policies, 
eligibility criteria, and procedures 

* assess program applicants for 
eligibility, help client complete 
application forms, etc. 

° determine type of device/supplies 
that best suit client’s need 

° provide client with list of Registered 
Vendors in his or her area 


* discuss client's equipment needs and 
technical support requirements with 
vendor 

° ensure that client receives appropriate 
assessment and trial equipment from 
vendor 


° follow up with client to ensure that 
correct authorized equipment has 
been delivered and client’s needs 
are being met by prescribed device/ 
supplies 


¢ meet all conditions specified in the 
Vendor Agreement 

* maintain up-to-date knowledge of 
Program-listed equipment 

° keep adequate stock of equipment it 
is authorized to sell 

° educate client and authorizer on 
makes and models of equipment 
available and maintenance it requires 

* provide reasonable variety of 
assessment equipment for client to try 
when requested by authorizer 

* work with client and authorizer to 
ensure that equipment meets the 
individual’s needs 

* provide required price quotes to client 
and Program 

* notify authorizer when equipment has 
been delivered to client so authorizer 
can follow up 

* honour manufacturer warranties and 
provide after-sale service 
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e establishing a standard processing time for 
claims and a standard response time for tele- 
phone and written inquires, and monitoring 
timeliness and help-desk effectiveness against 
those standards. 

@® reporting to management on backlog and 
workload statistics for data entry and claims 
assessment in each device category. This has 
helped to prioritize and assign work, and, at 
the time of our audit, the backlog had largely 
been addressed. Claims in all device categor- 
ies were being entered and adjudicated within 
the Ministry’s standard timeframe of six to 
eight weeks. 

e developing procedures for investigating 
complaints and maintaining records of the 
number and nature of complaints. 

@ re-instating standing committees in response 
to one of our 2001 audit recommendations. 
The committees meet twice a year to provide 
advice on policy, eligibility criteria, and pro- 
gram evaluation and monitoring strategies. 
There are currently four committees on the 
major device categories: mobility, prosthet- 
ics and orthotics, respiratory, and sensory. 
Committee members include health-care 
professionals, vendors, manufacturers, and 
consumers from across the province. 

@ conducting customer satisfaction surveys 
every two years to assess the level of client 
satisfaction and to improve service delivery 
models. Three surveys were completed since 
the 2002/03 fiscal year. More than 85% of 
respondents said that, overall, they were satis- 
fied with the Program. Some respondents, 
however, said that they were concerned about 
the reasonableness of the amounts they had to 
pay for devices or supplies. 


Program Cost Effectiveness 


While the Ministry has improved its service delivery 
to clients, it has not focused enough attention on 
ensuring that the Program is being delivered as 


cost-effectively as possible. As outlined in the fol- 

lowing sections of our report, we believe there are 
a number of areas where more rigorous oversight 
would yield significant savings. 


PRICING 


In 2004, the Program implemented a new pricing 
approach called the fixed pricing model, under 
which vendors are not allowed to charge more 

than the Program-approved prices. At the time of 
implementing this new approach, the Program- 
approved prices had not been adjusted since 1993. 
The goal of the Program’s pricing policy is to ensure 
that prices are fair, consistent, and equitable across 
device categories. To achieve this goal, the Ministry 
is required to regularly review and update the 
prices it has set for the devices and supplies that 
the Program covers. Home oxygen is an exception 
because its prices have been fixed on the basis of a 
contractual pricing agreement with vendors. 


Pricing of Home Oxygen 


Reasonableness of Pricing 
As illustrated in Figure 4, there are three different 
methods of providing home oxygen to clients: liquid 
oxygen, concentrators, and cylinders. The cost 
is not the same for all three methods. In general, 
liquid oxygen is the most expensive because of 
the high service costs associated with refilling and 
replacement. 

The Ministry currently pays directly to home 
oxygen vendors a single rate of $389 per month per 
client, with a $25 premium for clients in northern 
areas. Instead of using competitive open tendering, 
the price was set on the basis of an agreement 
negotiated with vendors (see Compliance with 
Negotiated Pricing Agreements). Because Ontario 
currently pays a fixed monthly rate for delivery of 
home oxygen regardless of the method used, the 
Ministry indicated that it did not track oxygen use 
by delivery method. In its response to our follow-up 
report in 2003, however, the Ministry indicated 


ee ee eee ee 
Figure 4: Types of Oxygen Systems 


Source of information: Ministry of Health and Long-Term Care 


Types of 


Oxygen 


Systems Description 


concentrator ° plug-in machine that extracts and 


accumulates oxygen from the air in a 
room 


* does not need to be replaced regularly 
or refilled, because it continually extracts 
existing oxygen from the air 


cylinder * cylindrical tank storing compressed 


oxygen 
* large tanks are used inside the home, 
and small tanks are used during outings 


or travel 
SDR Ue FE IE ae Nee EA ea SW eh Le OS 


liquid * stores oxygen in liquid form in large 
Stationary containers called reservoirs 


* liquid is turned into gas before it leaves 
the container 


* portable units for use during outings or 
travel are filled from the reservoir 


that it intended to design a system for collecting 
statistics on the use of liquid oxygen and concentra- 
tors in preparation for negotiating the next pricing 
agreement. 

Our review of invoices from vendors showed 
that the majority of clients were on a concentra- 
tor system. One major vendor indicated to us that 
almost 90% of its clients were on a concentrator 
system. Unlike cylinder or liquid oxygen systems, 
concentrators do not need to be refilled with 
oxygen, nor must they be replaced often. They are 
simply plugged in and start accumulating and deliv- 
ering a continuous stream of oxygen from the air in 
a room. According to the manufacturers’ invoices 
the vendors provided, the cost of a concentrator 
could range from $400 to $1,000. Concentrators 
generally last from five to seven years. Yet the total 
revenue that a vendor receives from the Program 
for a concentrator that lasts five years is approxi- 
mately $23,000 ($389 x 12 months x 5 years). 

The Ministry advised us that most of the $23,000 
relates to other ongoing client services that are not 
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directly related to the cost of the concentrator or 

routine maintenance. However, the Ministry has 

not assessed whether this is a reasonable amount 
for it to be paying nor compared it to what other 

provinces are paying for a similar service. 

We noted that, although the Ministry had not 
done any cost analyses for the three different 
methods of delivering home oxygen, it had con- 
ducted a cross-jurisdictional study of home oxygen 
programs. However, the Ministry could not draw 
any meaningful comparisons from its study because 
it did not know how home oxygen delivery was 
divided up among the three different methods in 
each jurisdiction. 

According to the Ministry’s Home Oxygen Joint 
Utilization Committee, Alberta is the most compar- 
able jurisdiction to Ontario, and Ontario’s home oxy- 
gen prices are at the “high end” compared to other 
jurisdictions. We noted that Alberta’s rate is $331 per 
month, 18% lower than Ontario’s rate of $389. 


Compliance with Pricing Agreements 
The monthly rate for home oxygen was fixed on 
the basis of a pricing agreement negotiated with 
vendors represented by the Ontario Home Respira- 
tory Services Association (OHRSA). The agreement 
was signed in 2004 after the Ministry requested an 
exemption from the competitive open tendering 
requirement of the Management Board of Cabinet’s 
Procurement Directive for Goods and Services. The 
Ministry also requested approval for negotiations 
that would maintain annual program expenditures 
at $54.6 million for four years. The request indi- 
cated that if utilization increased by more than 3%, 
the Ministry would be able to lower the set price 
by 3% per year. With the approval of the Manage- 
ment Board of Cabinet, the Ministry negotiated and 
signed the agreement with OHRSA to maintain pro- 
gram expenditure on home oxygen at $54.6 million 
per year to March 31, 2008. The final agreement, 
however, did not contain any terms for a price 
reduction based on an increase in utilization. 
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For each of the fiscal years from 2004/05 to 
2007/08, annual expenditures were $6 million 
to $11 million more than $54.6 million, or $33 mil- 
lion more in total. Ministry staff informed us 
that they felt the agreed-upon yearly amount of 
$54.6 million did not take into account the continu- 
ally growing aging population and the prevalence 
of chronic obstructive pulmonary disease (COPD), 
a respiratory disease primarily caused by tobacco 
smoke. The Ministry further indicated that it was 
its understanding that the $54.6 million could 
be exceeded if utilization increased significantly. 
The Ministry confirmed that in those years where 
expenditures exceeded the contract amount, the 
excess amounts were approved through Treasury 
Board Orders, which authorize an increase in 
program expenditures if the increase is offset by a 
corresponding reduction of expenditures in another 
program area. 

Our review found that the submission to the 
Management Board did not make reference to the 
annual limit of $54.6 million previously imposed 
by the Management Board or the existence of such 
a limit in the agreement with the vendors. The 
Ministry also advised us that it felt that the Manage- 
ment Board had been apprised of this during in-year 
updates, although there was little documentation to 
indicate this issue had been specifically raised. 

In 2008, the Ministry sought and received 
approval to negotiate an extension of the existing 
agreement and a continued exemption from 
competitive tendering. The Ministry extended 
the existing agreement to 2009, with an option to 
renew for another year. We believe some clarifica- 
tion is needed with respect to whether continued 
exemption from competitive tendering is condi- 
tional on total annual expenditures not exceeding 
$54.6 million. 

In addition, we found that the Ministry paid cer- 
tain vendors the $25 premium for clients in north- 
ern areas even though, according to the agreement 
with the vendors, these clients were not eligible for 
the premium. This has resulted in potential overpay- 
ments of approximately $2.2 million from 2002/03 


to 2008/09. When we brought this to the Ministry’s 
attention, it noted that the agreement was in error 
because it was inconsistent with the intent of 
program policy. The Ministry is working with the 
vendor community to correct the agreement. 


__ RECOMMENDATION 1 


To ensure that prices for home oxygen are com- 
petitive, the Ministry of Health and Long-Term 
Care should perform a more rigorous analysis 
of the costs of delivering home oxygen under 
each method before negotiating the new rate 
for home oxygen. This analysis should consider 
the oxygen prices other provinces are paying to 
ensure that Ontario is getting good value, espe- 
cially given the economies of scale that should 
result from being the largest province. 

The Ministry should seek clarification 
from the Management Board of Cabinet with 
respect to the approval not to tender for home 
oxygen provided that “total expenditures for the 
program should not exceed $54.6 million annu- 
ally”. Specifically, it should confirm whether the 
maximum can be exceeded due to an increase in 
utilization provided the increase can be funded 
internally within the Ministry and approved 
through a Treasury Board Order. 





The Ministry is conducting an open and trans- 


parent procurement process to establish a 
vendor-of-record list for the provision of home 
oxygen services over the next five years, with 
the option to extend for up to two years. Reim- 
bursement rates are under review, and the Min- 
istry has retained the services of an independent 
consultant to provide expert advice on deter- 
mining a fair price. The consultant conducted 
interviews with key health-care experts, studied 
the drivers that affect the cost of home oxygen 
services as well as how these cost drivers might 
change over the next seven years, and reviewed 
pricing models in other jurisdictions. , 


In 2008, the Ministry received approval to 
negotiate a one-year agreement with a possible 
one-year extension with home oxygen vendors. 
The approval noted the forecasted expenditures 
on home oxygen in 2008/09 and 2009/10 as pro- 
jected based on current utilization growth rates. 

The Ministry firmly believes that it has 
sought and received the appropriate approvals 
for program spending in all instances where 
expenditures exceeded the initially approved 
allocation, and the Ministry has approval to 
establish a Vendor of Record list and the pricing 
for home oxygen services. In addition, the Min- 
istry will seek clarification from Treasury Board 
and Management Board Secretariat. , 

The Ministry’s payments of the $25 premium 
for clients in northern areas were made cor- 
rectly despite incorrect wording in the vendor 
agreement and did not result in a potential 
$2.2-million overpayment. 


Pricing of Other Devices 


According to the mark-up policy outlined in the 
Policies and Procedures Manual for the Assistive 
Devices Program, “the price for a product should be 
the manufacturer’s unit cost to the vendor for that 
product plus a reasonable return up to 33.3%. The 
result will be a vendor margin of 25%.” Ministry 
staff informed us that the purpose of this policy is 
to ensure that Program-approved prices are reason- 
able, appropriate, and consistent with fair market 
value. Hearing aids are an exception. Vendors of 
these devices are not allowed to mark them up at 
all. The price a vendor charges for a hearing aid 
device must be the same as that of the manufac- 
turer. See Figure 2 for a detailed explanation of 
how pricing works in each device category. 


Mobility Aids 
We noted from our sample testing that the cost 
for mobility aids varied significantly from vendor 
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to vendor, largely based on the size and buying 
power of each individual vendor. The average 
mark-up between the Program-approved price and 
the vendor cost was 84%, which was significantly 
higher than the 33.3% set out by the Program as 
reasonable. Our testing indicated that: 
@ the Ministry set the prices on the basis of 
the cost of a single unit, without taking into 
consideration the volume discounts vendors 
would normally get when purchasing multiple 
units of the same device; and 
@ the Ministry conducted pricing reviews in 
2004 and 2006, but has not done one since 
although they are required every two years: 
therefore, current prices may not reflect pos- 
sible decreases in manufacturers’ unit costs 
because of technological advances. 


Hearing Aids 
As noted in Figure 2, program funding covers the 
cost of the hearing aids, ear moulds, options/acces- 
sories listed with the Program, plus the applicable 
dispensing fees charged by dispensers for duties 
such as ordering, fitting, and adjusting, and for 
instructing clients how to use hearing aids and care 
for them, but these fees cannot be for more than the 
amounts stipulated by their professional associa- 
tions’ fee schedules. We selected a sample of claims 
to assess vendors’ and dispensers’ compliance with 
program policies and procedures and the reason- 
ableness of prices. We found that: 

e Vendors are not allowed to mark up the price 
of hearing aids. The price they charge must 
be the same as the manufacturer’s price. 
However, we noted cases where vendors did 
not adhere to this requirement. For example, 
a vendor charged a 50% mark-up of about 
$430. The vendor explained to us that the 
mark-up was for the “worry-free” program, 
but this was not apparent on the invoice. 
Another vendor did not pass on the savings to 
clients when manufacturers’ discounts were 
obtained by buying hearing aids in bulk. This 
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vendor consistently charged a 30% mark-up 
amounting to about $200 on top of the manu- 
facturer’s price. 

e The Ministry does not check if dispensers 
are complying with fee schedules before it 
approves claims. The Ministry only examines 
dispensing fees if a vendor’s claim is selected 
for review by the Compliance and Quality 
Assurance Unit. We noted instances where 
dispensers have been consistently billing a dis- 
pensing fee higher than the program average, 
which is about $650. In one case, a dispenser 
had an average dispensing fee of more than 
$1,700, which would result in the client over- 
paying his or her share of the cost because the 
maximum amount the program funds is $500 
per hearing aid. 


Respiratory Devices 
More than 90% of program funding for respiratory 
devices helps pay for Continuous Positive Airway 
Pressure systems (CPAPs), which help people with 
obstructive sleep apnea symptoms breathe easier 
during sleep. 

We reviewed a sample of invoices and noted 
that the average mark-up between the Program- 
approved price and the vendor cost of CPAPs was 
117%, much more than the mark-up of 33.3% set 
out by the Program as reasonable. Many vendors 
claimed that the Program’s mark-up policy does 
not take into account the additional indirect sup- 
port costs of providing CPAP therapy to clients, 
such as set-up time, client visits, and maintenance. 
However, according to the Policies and Procedures 
Manual for the Assistive Devices Program, the 
Program-approved price is not intended to cover 
support or service fees but rather only the complete 
system, which consists of a CPAP device, a heated 
humidifier, a basic mask and headgear, a carrying 
case, six feet of tubing, the necessary caps and 
filters, a power cord, and an instruction manual. 
Ministry staff informed us that some vendors have a 
higher mark-up than is allowed because they obtain 


savings by purchasing devices in bulk, yet this is 
not taken into consideration when the Ministry 
establishes the Program-approved prices. The Min- 
istry advised us that it is currently in the process of 
conducting a review of CPAP device prices. 
Vendors may offer to clients extra items or 
services that are not covered by the Program, such 
as service packages. The Ministry requires vendors 
to provide clients with itemized invoices for the 
additional services, and to explain to clients that 
they have the option to purchase only the Program- 
funded device if they wish. Our review of vendors’ 
invoices to clients revealed that most clients were 
charged for additional items that were not covered 
by the Program. We were not able to confirm 
whether the vendors informed clients that they 
could choose to purchase only the Program-funded 
device, but we did note cases where they did not 
provide the required itemization on their invoices. 
Instead, the invoices showed a lump sum and sub- 
tracted the portion covered by Program funding. 
For instance, the invoice listed a charge of $1,600 
for a “CPAP package” and subtracted the Program’s 
$780 portion from this amount without providing 
any cost breakdown for the remaining portion 
that the client had to pay. We are concerned that 
ambiguous invoices may lead clients to mistakenly 
believe that the total price is for the basic device 
only. Ministry staff indicated that they have similar 
concerns and have begun to look into this issue. 


Communication and Visual Aids 
The Program funds the purchase of computer 
equipment to be used as communication aids 
or aids for the visually impaired. The Ministry 
informed us that it had done two pricing reviews, 
one in 2004 and one in 2006, but our review indi- 
cated that the Program-approved prices still appear 
to be significantly higher than fair market value. For 
example: 
@ We reviewed a sample of complaints and 
noted that excessively high prices for com- 
puter equipment liave been a recurring issue. 


The complaints revealed that vendors often 
charged inflated prices for computer equip- 
ment. Some clients found the same devices 
at much lower prices from vendors not regis- 
tered with the Program. In one case, a client 
complained that the price quote a vendor 
provided was more than $4,100, nearly three 
times the price the client found online for the 
same device. 

e@ We obtained price quotes from five Program- 
registered vendors for the same computer sys- 
tem with a comparable monitor and printer. 
The prices they quoted ranged from $1,300 
to $4,400. The vendor that quoted $4,400 


offered to cover the client’s 25% portion of the 


cost ($1,100). 

We reviewed a sample of claims from major 
vendors for computer equipment and related sup- 
plies. We noted the following questionable pricing 
practices: 

e Prices of the computer equipment had been 
marked up much higher than the program 
maximum of 33.3%. In fact, the average 
mark-up was 128%. Component parts such 
as monitors, printers, and scanners had the 
highest mark-ups. For example, the Program- 
approved price for a monitor is $1,332, and a 


vendor can often obtain a comparable monitor 


for only $250, which amounts to a mark-up 
of more than 400% if the vendor sells it for 
the Program-approved price. Ministry staff 
acknowledged that Program-approved prices, 
last reviewed in 2006, probably exceed cur- 


rent fair market prices, and that vendors could 


therefore obtain returns greater than 33.3%. 
The Ministry indicated that it would deter- 
mine appropriate prices for computers as part 


of its pricing review in the 2009/10 fiscal year. 


e@ We noted some instances where vendors 
billed the Program separately for two devices 
(a printer and a scanner), but only supplied 
the client with one device (an “all-in-one” 


printer). In 2006, the Ministry had also identi- 


fied this issue in its review of a vendor and 
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subsequently referred the case to the Ontario 
Provincial Police. Ministry staff acknowledged 
that this practice was improper, but explained 
that they lack the resources to thoroughly 
review vendors and discourage such practices 
from recurring. 

@ We noted some cases where vendors added 
service fees to the Program-approved price. 
One vendor required that clients sign an 
agreement indicating that a service fee of 
about $700 was included in the total the 
vendor had charged the Program. Ministry 
staff confirmed that other fees such as service 
charges are not supposed to be added to the 
fixed Program-approved price. 


RECOMMENDATION 2 





To ensure that the cost of equipment paid for 

by the Ministry and its clients is competitively 

priced, the Ministry of Health and Long-Term 

Care should: 

e@ conduct regular pricing reviews for each 
device category and update Program- 
approved prices accordingly; and 

@ take volume discounts and technological 
advances into consideration when updating 
Program-approved prices. 


The Ministry’s policy is to review prices every 
two years. As such, the Ministry will ensure 
that pricing reviews occur on a timely basis. 

A pricing review was initiated in 2008 and is 
scheduled to be completed in the 2009/10 fiscal 
year. 

The Assistive Devices Program works to 
ensure that prices across device categories are 
fair, consistent, and equitable. The Program’s 
funding model is also expected to take into con- 
sideration the economic and social environment 
within which the Program receives its share 
of public funds, and to enable clients to access 
needed devices. Prices set through the Pricing 
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Policy must therefore be suitable for clients 
regardless of their location and their access 
to larger vendors that may have a purchasing 
advantage over small and remote vendors. 


VERIFICATION AND REVIEW PROCESS 
Monitoring of Claims 


Home Oxygen Claims 
Home oxygen applicants must meet specific 
eligibility criteria. Their eligibility is determined 
by the results of an arterial blood gas test or an 
oximetry test, both of which measure oxygen levels 
in the blood. In response to our audit in 2001, the 
Ministry changed the eligibility testing intervals 
in October of that year. Before that time, a person 
Was required to submit results of an arterial blood 
gas test and to reapply annually for continued 
coverage by submitting the results of an oximetry 
test. Individuals are now required to be assessed on 
three separate occasions: the results of an arterial 
blood gas test must be submitted upon their initial 
application; the results of an oximetry test must 
be submitted three months afterwards; results 
of another oximetry test must be submitted 12 
months after the initial application. Although no 
further submission of clients’ test results is required 
after the third assessment, the policy outlined in 
the Program’s Administration Manual for Home 
Oxygen states, “clients are required to have their 
oxygen requirements assessed annually once long- 
term funding assistance has been provided.” These 
annual assessments were done by respiratory ther- 
apists employed by the vendors. 

We reviewed a sample of client files from two 
major vendors. These vendors account for more 
than 60% of the home oxygen supply that is funded 
by the Program. We noted that more than one-third 
of the files showed that either no assessments had 
been done for the past 18 months, no test results 
had been recorded, or the results indicated that 
the clients no longer met the criteria for long-term 


home oxygen supply. Perhaps not surprisingly, the 

vendors had not advised the Ministry of this—even 
in the cases where test results indicated home oxy- 
gen was no longer required. We also noted that: 

e [tis not clear who is responsible for discon- 
tinuing home oxygen supply for clients who 
no longer meet the eligibility criteria. Vendors 
told us that it is not their responsibility, even 
if they are aware that a client no longer meets 
the eligibility criteria. They indicated that 
only a physician could recommend discon- 
tinuing home oxygen. 

e Ministry staff had also identified cases where 
long-term clients were receiving home oxygen 
even though they no longer met the eligibility 
criteria. In a report to program management, 
program staff recommended that clients sub- 
mit the results of reassessments on a regular 
basis, but the Ministry has not yet taken any 
specific action to resolve this issue. 

According to a cross-jurisdictional study the 
Ministry did in 2008, Ontario had among the 
largest proportion of home oxygen users of all 
the provinces: 150 users per 100,000, compared 
to the Canadian national average of 60 users per 
100,000. Alberta requires more frequent and strin- 
gent assessment of home oxygen needs than does 
Ontario. During their first year of home oxygen use, 
clients in Alberta are required to be assessed three 
times with arterial blood gas tests. After that, they 
must be reassessed every six months to show that 
they still warrant home oxygen. 

In Ontario, respiratory therapists employed by 
home oxygen vendors assess clients with oximetry 
tests. In other provinces, such as British Columbia 
and Saskatchewan, independent respiratory ther- 
apists at the Regional Health Authorities conduct 
oximetry testing. The Ministry informed us that 
Ontario’s health-care system differs from that of 
other provinces with respect to the distribution of 
respiratory therapists in the community, and that 
Ontario currently does not have enough respiratory 
therapists working independently from vendors. 
The obvious risk associated with vendor-employed 


respiratory therapists assessing clients for home 
oxygen eligibility is that it is in the vendor’s interest 
for the client to continue to receive home oxygen. 


RECOMMENDATION 3 


To ensure that funding for home oxygen is 
provided only to individuals who require it for 
medical reasons, the Ministry of Health and 
Long-Term Care should: 

@ assess whether more stringent vendor 
oversight is required to ensure that the 
required periodic assessment tests are being 
appropriately conducted and reported, or, 
alternatively, consider the practicality of 
having independent respiratory therapists 
perform eligibility assessments, rather than 
vendors’ staff; and 

e establish procedures and assign clear 
responsibility for discontinuing home oxygen 
supply to clients who no longer meet the 
medical eligibility criteria. 


Respiratory therapists are regulated health 
professionals who are required to meet the 
standards of practice established by the College 
of Respiratory Therapists of Ontario. Their 
employment by home oxygen vendors does not 
mitigate their requirements to meet the stan- 
dard of practice of their profession. 

Home oxygen therapy is provided only to 
individuals who require it for medical reasons. 
The Ministry requires an assessment and 
prescription by a qualified physician, and the 
prescribed service continues until the physician 
deems it unnecessary on the basis of the indi- 
vidual’s clinical needs. The Ministry will require 
annual written confirmation of the patient’s 
continuing need for home oxygen therapy. 
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Mobility Aids—Scooter Claims 
The Program funds power scooters, which are a 
type of mobility aid. Individuals are only eligible 
for scooters if they require them to meet long-term 
basic and essential mobility requirements; do not 
require specific postural support now or in the 
future; do not intend to use the scooter to replace a 
car or other mode of transport; and can get on and 
off the scooter without assistance. 

In the 2004/05 fiscal year, the Ministry con- 
tracted with a third party to review scooter claims 
and found some clients who had been authorized 
for scooters did not meet the eligibility require- 
ments. We noted that total scooter claims decreased 
by 13% from 2004/05 to 2005/06, the year after 
the Ministry’s review. The review’s deterrence 
effect did not last very long, however—we noted an 
increase in scooter claims of 109% from 2005/06 to 
2008/09 (see Figure 5). 

More than 150 vendors received program 
funding for power scooters in the 2007/08 fiscal 
year. We reviewed the top ten of these vendors 
and selected those with at least a 200% one-year 
increase in scooter claims. We found three vendors 
whose 2008/09 scooter claims had increased by 
more than 800% (from $88,000 to $805,000) com- 
pared to three years ago (see Figure 6). 
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One of the vendors had gone into business only 
four years ago, so the Ministry’s last review of 


Figure 5: Scooter Claim Trend, 2004/05-2008/09 
($ million) 


Source of data: Ministry of Health and Long-Term Care 
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Figure 6: Scooter Claims of Sample Vendors from 
2004/05-2008/09 ($ 000) 

Source of data: Ministry of Health and Long-Term Care 
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scooter claims pre-dated it. The other two vendors 
had been selected by the Ministry for a review in 
2004/05, because of unusual claim patterns. The 
Ministry’s review resulted in the termination of 
the agreement with the authorizer associated with 
these two vendors for authorizing scooters for cli- 
ents who were not eligible for program funding, but 
the Ministry did not report its concerns regarding 
the authorizer’s actions to the relevant regulatory 
college. 

As noted in Figure 6, the Ministry’s 2004/05 
review of scooter claims had a deterrence effect, but 
this effect was short-lived. If the Ministry does not 
maintain a vigilant monitoring effort, it is unlikely 
to deter abuses of program funding for scooters. 
Ministry staff told us that they are planning a 
follow-up review of vendors with unusual scooter 
claim trends in 2009/10. 


Hearing Aids—FM System Claims 
FM systems make the speech signal more pro- 
nounced than background noise. They have been 
standard equipment for many years in educational 
settings for school-age children with hearing loss. 
To qualify for program funding for an FM system 
(or any hearing aid device the Program covers), 
an individual must have a long-term documented 
hearing loss that necessitates the use of an FM 


system as part of his or her daily activities for more 
than six months. 

Our review showed that claims for FM systems 
have risen significantly since 2004/05, especially in 
the senior age group (66 and over), whose claims 
increased by almost 1,800%, from 187 claims or 
$250,000 in 2004/05 to 3,557 claims or $4.8 mil- 
lion in 2008/09 (see Figure 7). The Ministry 
became aware of this issue in October 2008 when 
following up on a complaint. In January 2009, it 
developed a plan of action to identify improper 
claims and prevent further abuses. Ministry staff 
indicated that they had taken action to strengthen 
the review process for FM systems, such as requir- 
ing pre-approval for FM-system funding for adults 
and establishing a special committee to develop 
new eligibility criteria. However, the Ministry’s 
actions could have been more timely, given that 
claims began to increase significantly more than 
three years ago. 

We also noted that, in some cases, a manufac- 
turer of FM systems offered a rebate to vendors 
for a Program client’s 25% portion of the bill. The 
rebate would be in the form of a coupon or discount 
on the vendor’s next purchase. This gave vendors 
an incentive to sell FM systems, and clients, who 
were getting them for no cost, had no reason to 
refuse the offer. The Ministry has also identified 
SS ee ee eee ee 
Figure 7: Number of FM Systems Claims by Age Groups 
from 2002/03-2008/09 


Source of data: Ministry of Health and Long-Term Care 
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cases where vendors told clients that an FM system 
would come as part of a “package” with the clients’ 
hearing aids. When asked, the clients told the Min- 
istry that they never used the FM systems. 


Ostomy Supply Claims 
Individuals with permanent ostomies (surgical 
openings made necessary by the loss of normal 
bladder or bowel function) are eligible to receive 
a grant of $600 per year for each ostomy, up to 
a maximum of two ostomies, for the purchase of 
related supplies. 

In addition to sending letters to 2% of the 
physicians who apply for ostomy grants on behalf 
of their patients—to confirm their eligibility—the 
Ministry has occasionally conducted reviews of 
ostomy supply grants to ensure that clients have 
used their grant payments for the intended purpose 
and that they still qualify for the grant. The Min- 
istry’s last review, done in 2005, examined ostomy 
claims from 2001 to 2004. Only 40 of the 287 
clients under review were able to provide receipts. 
They indicated that either the Ministry told them 
that they did not have to provide receipts or they 
were not aware that they had to keep their receipts. 
Even though the 2005 review results indicated 
significant compliance problems, ostomy claims 
have not been reviewed since because of staff con- 
straints. Ministry staff informed us that they would 
re-instate the review process and would instruct 
clients to keep their receipts. 


Insulin Pump and Supply Claims 
The insulin pump and supplies program was imple- 
mented in December 2006. Ontario was the first 
Canadian jurisdiction to fully fund insulin pumps 
for children and youth (age 18 and under) with 
type 1 diabetes, although Saskatchewan, New- 
foundland and Labrador, and British Columbia now 
offer similar coverage. In September 2008, Ontario 
extended program coverage for insulin pumps and 
supplies to adults with type 1 diabetes. 
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We reviewed a sample of claims for insulin 
pumps and noted cases where the delivery date of 
the pump preceded the date the client’s eligibility 
for the pump was assessed by a physician. The 
policy in the Program’s Administration Manual 
for Insulin Pumps and Supplies is that “insulin 
pumps must be purchased after the client has been 
assessed by physician. Otherwise, the insulin pump 
will not be considered for funding. Clients who 
purchase an insulin pump prior to the assessment 
cannot then submit an application form and expect 
reimbursement from the Program.” We suggested 
to the Ministry that it may want to re-examine the 
current policy, but if it is deemed appropriate it 
should be enforced. 





" RECOMMENDATION 4 





To ensure that Assistive Devices Program fund- 

ing for devices and supplies is provided only to 

individuals who are eligible for it, the Ministry 

of Health and Long-Term Care should: 

e identify and investigate abnormal claim pat- 
terns through regular reviews; 

@ take action to deter authorizers or vend- 
ors that who are suspected of abusing or 
misusing program funding, including sus- 
pending their registration with the Program 
and bringing the matter to the attention of 
the appropriate regulatory college or profes- 
sional association where professional miscon- 
duct is suspected. 





The Ministry notes that authorizers receive no 
funding from the Assistive Devices Program. 
The Ministry agrees that it must continue to 


take actions to deter abuse and misuse of pro- 
gram funding and provide training and infor- 
mation to authorizers and vendors regarding 
program requirements. 

In 2008, the Ministry received approval to 
develop a new information system to replace 
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its current legacy system; the new system is 
expected to be implemented in spring 2011. Sys- 
tem re-development will support the Program by 
enhancing monitoring capacity. The new system 
will also help the Ministry to monitor patterns 
and trends of authorizer and vendor activity. 
The Ministry will improve its statistical 
reporting to ensure that abnormal claim pat- 
terns are identified and appropriate actions 
are taken. The Ministry will also liaise with the 
appropriate regulatory colleges to determine 
contacts and protocols. 


Post-payment Review Process 


In response to our 2001 audit, the Ministry re- 
established its post-payment review process in the 
2002/03 fiscal year. The objective of this process is 
to ensure economic, efficient, and effective oper- 
ation of the Program; correctness and validity of 
claims paid; and compliance with program policies 
and procedures. The Ministry also expanded its veri- 
fication process to cover all major device categories 
rather than just home oxygen and ostomy grant 
recipients, which was the case in our last audit. Min- 
istry staff indicated that they use a risk-based review 
approach that focuses on areas where irregularities 
are prevalent, are expected to occur, or would 

result in substantial financial loss to the Program. 
Although the Program has completed 138 reviews 
and has identified about $2 million in recoverable 
overpayments since the 2002/03 fiscal year, we 
have concerns regarding review resources, coverage, 
and selection, as noted in the sections below. 


Review Resources and Coverage 
The Ministry currently has three compliance-and- 
quality-assurance staff to monitor the activities of 
more than 1,000 vendors and 6,000 authorizers. 
They conduct two types of reviews: desk reviews 
and field reviews. Desk reviews are performed 
in-house without any on-site inspection. A field 


review is required only if material discrepancies 
are observed in a desk review. We noted that, of 
the 138 reviews completed since the 2002/03 fiscal 
year, only 22 were field reviews. We were informed 
that the number and the extent of reviews were 
limited by the resources available. Not only are 
hundreds of millions of dollars paid out annually, 
but expenditures have increased by more than 90% 
between 2001/02 and 2008/09. Yet the number of 
compliance-and-quality-assurance staff has been 
the same since 2002. 

Although only 23 reviews were completed in 
2008/09, they successfully identified overpayments 
of about $600,000. The high rate of overpayment 
identified by even a limited number of reviews sug- 
gests that expanded review resources are justified 
from a purely financial payback perspective and, if 
combined with a communication strategy, would 
send a clear message that inappropriate authoriz- 
ing and billing practices will not be tolerated. 


Review Selection 
As noted above, the Ministry’s audit selection 
process is supposed to target vendors that are at 
the highest risk of abusing the program, because 
the Ministry has limited compliance-and-quality- 
assurance resources. At the time of our audit, the 
Ministry had the capability to extract data from 
the assistive devices database according to specific 
risk-factors, but it was not using this capability in 
a systematic way. Our audit identified a number of 
high-risk areas that warrant more regular review 
effort (see sections on Monitoring of Claims and 
Conflict of Interest). We also felt that a lack of 
training on risk assessment partially accounted 
for deficient monitoring. We were informed that, 
although front-line staff such as claims assessors 
and program co-ordinators are responsible for 
informing compliance-and-quality-assurance staff if 
they observe irregularities, the front-line staff have 
received no formal training on risk-assessment tech- 
niques to identify “red flags” indicating potential 
fraud or misconduct. The Ministry informed us that 


it would be working to improve awareness of fraud 
risks in staff's day-to-day roles by developing a com- 
prehensive training program on the risk-assessment 
process early in the 2009/10 fiscal year and imple- 
menting a risk-assessment tool in summer 2009. 


Fraud Investigation 


The Program co-ordinates with the Ministry’s 
Fraud Programs Branch (which became part of the 
Accounting Policy and Financial Reporting Branch 
after we completed our audit work) to refer poten- 
tial cases of fraud to the Ontario Provincial Police 
(OPP). Since 2001, the Program has identified and 
referred 19 such cases to the OPP. We noted that, of 
the $1.8 million that has yet to be recovered, more 
than $900,000 involves two vendors that were 
referred to the OPP shortly after the start of our 
audit fieldwork in 2009. The Ministry indicated that 
it was only able to recover $43,000 out of $1.8 mil- 
lion because it has to wait for the OPP to complete 
its investigations and referrals to the court, through 
which restitution is to be made. 

The referral and investigation process can take 
a long time. In our review of cases for which the 
investigations had been completed, we noted that 
they took on average about 530 days from the date 
of referral to completion. The Ministry can termin- 
ate the registration status of vendors and authoriz- 
ers if there is any violation of their agreements and/ 
or deviation from program policies not corrected to 
the satisfaction of the Program. However, during 
the investigation period in the above cases, the 
vendor continued to submit claims and bill the Pro- 
gram. Ministry staff told us this had been a matter 
of some concern to them, but they felt they could 
not take action until the OPP had completed its 
investigation. The Ministry also has the obligation 
to report authorizer misconduct to the respective 
professional colleges and associations, which have a 
strong incentive to maintain the good reputation of 
their membership and to protect the public. How- 
ever, we noted that the Ministry has rarely taken 
such action. 


Assistive Devices Program 


RECOMMENDATION 5 


To more effectively identify abuses, recover 
overpayments, and deter misconduct, the Min- 
istry of Health and Long-Term Care should: 

@ expand its efforts and resources to better 
monitor vendors’ and authorizers’ compli- 
ance with program policies and procedures; 

@ take timely corrective action to terminate 
agreements with vendors and authorizers 
who have clearly violated program policies; 

@ work with the Ministry’s Accounting Policy 
and Financial Reporting Branch to elevate 
staff risk-awareness and risk-assessment 
skills; and 

e@ where there is clear evidence of potential 
misconduct, report its concerns to the appro- 
priate regulatory associations or colleges, 
which are responsible for ensuring the public 
is protected. 





The Ministry agrees and is reviewing its policies 


and procedures to ensure that there is clarity 
on eligibility criteria, pricing, and charges to 
clients. As well, the Ministry will continue to 
educate vendors and authorizers on program 
policies and procedures, and will terminate 
agreements with vendors and authorizers who 
have acted fraudulently. 

The Program is working with the Account- 
ing Policy and Financial Reporting Branch to 
complete this work in the 2009/10 fiscal year 
and to provide staff with the risk-management 
skills and tools they need to help them more 
rigorously manage vendor and authorizer 
agreements. The Ministry will also liaise with 
the appropriate regulatory colleges to determine 
contacts and protocols. 
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CONFLICT OF INTEREST 


The Program considers it is a conflict of interest 
whenever there is a financial relationship between 
an authorizer and a vendor (see Figure 3 fora 
description of authorizers’ and vendors’ roles and 
responsibilities). The Policies and Procedures 
Manual for the Assistive Devices Program states that 
it would be considered a conflict of interest where: 

@® aphysician who prescribes a device for an 
eligible person has any financial relationship 
with the vendor selling that device; 

@ an authorizer who determines client eligibility 
refers clients to a specific vendor or receives 
any fee or benefit from a vendor, directly or 
indirectly; or 

@ avendor gives any fee or benefit, directly or 
indirectly, to a person who determines client 
eligibility or refers clients to that vendor. 

To ensure that clients are given a choice of vend- 
ors and to prevent conflict of interest, authorizers 
are required to provide clients with a list of vendors 
in their area rather than refer them to any one 
vendor. As a condition of their registering with the 
Program, vendors and authorizers are required to 
comply with the Program’s conflict of interest policy 
by signing agreements with the Ministry: 

e Inthe authorizer agreement, authorizers 

also agree not to influence eligible clients to 
purchase devices from any specific vendor, not 
to accept from any vendor payment in cash or 
kind (directly or indirectly) for recommending 
any device and/or their assessment services, 
and not to have a professional affiliation 

with a vendor. Failure to comply with these 
terms will result in the Ministry immediately 
revoking the authorizer’s registration with the 
Program. 

@ Inthe vendor agreement, vendors agree to 
conduct their businesses without conflict 
of interest as described in the Policies and 
Procedures Manual for the Assistive Devices 
Program. Breach of this provision will result 
in termination of the vendor agreement. 


Mobility Aids 


As noted in the Monitoring of Claims sections, three 
vendors had scooter claims that increased by more 
than 800% over three years. Our analysis of these 
three vendors indicated that each of them had more 
than 70% of their claims authorized by only one 

or two authorizers. These authorizers and clients 
were often not located near the respective vendors. 
In many cases, the clients were located over 30 
kilometres away. Clients typically purchase their 
devices from a vendor located near their homes; 
therefore, we questioned whether the authorizers 
had provided a list of vendors to the clients in all 
three instances. We noted that there were many 
other vendors located near the clients and author- 
izers in question. We suspect that the authorizers 
may have recommended these specific vendors, 
which would be a potential conflict of interest and a 
violation of program policy. 


Hearing Aids 


Applications for hearing aid funding must be signed 
by a prescriber (a physician or an audiologist) who 
confirms that the client has hearing loss. The appli- 
cation also requires the signatures of the author- 
izer, dispenser, and vendor. It is possible for these 
three roles to be fulfilled by one person, so to avoid 
a conflict of interest, the Program requires that each 
application must be completed and signed by two 
health-care professionals who are not financially 
dependent on any of the other signatories. 

This requirement, if implemented and mon- 
itored appropriately, would minimize the risk of 
conflict of interest. Yet we noted that the require- 
ment is often not being met. We selected a sample 
of vendors with high volumes of hearing aid claims. 
Our analysis found numerous cases of apparent 
conflict of interest. For example: 

e A vendor with multiple locations had claims 

totalling more than $10 million since 2000. 
One physician prescribed most of the claims 
coming from this vendor’s various locations. 


Our discussion with the Ministry indicated 
that it was aware of the potential conflict of 
interest in early 2000. The Ministry referred 
the case to the Ontario Provincial Police in 
2004 and again in 2009. The Ministry told us 
that it cannot terminate its agreements with 
the vendor and authorizer while the matter is 
under police investigation. 

Another vendor was registered with the 
Program as both an authorizer and a dispen- 
ser. Since 2002/03, one physician was the 
prescriber for 99% of the vendor’s claims, 

for a total of $900,000. The vendor and the 
physician were located at the same address. 
We noted that the vendor was renting office 
space from the physician. In our review of this 
vendor’s file, we also saw that this physician 
had acknowledged that he had been referring 
clients to this vendor for a long time. Ministry 
staff confirmed to us that this relationship 
could be reasonably considered a potential 
conflict of interest. Yet the Ministry has taken 
no further action. 

Since 2002/03, the total claims submitted by 
one vendor were about $1.3 million. We noted 
that, at the time this vendor registered with 
the Program, its business insurance was in 
the name of a physician. We also noted that 
this physician, who may have been related 

to the vendor given their same last name, 
prescribed more than 65% of the vendor’s 
claims. Ministry staff explicitly identified 
such a relationship as a potential conflict of 
interest, because it could be reasonably con- 
cluded that a vendor and a physician who are 
related could be sharing profits. However, this 
potential conflict of interest has never been 
investigated. 

Another vendor has two locations, which are 
25 kilometres away from one another. Since 
2002/03, 96% of the claims at both locations, 
for a total value of more than $1 million, were 
approved and co-signed by the same physician 
and authorizers, who were also co-owners 
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of the vendor’s business. The ongoing, close 
association among these parties would seem 
to indicate a potential conflict of interest, but 
the Ministry has never reviewed any of the 
parties involved. 

In cases where the Ministry did find potential 
conflict of interest or misconduct of health-care 
professionals, it seldom terminated the authorizer’s 
or vendor’s ability to authorize benefits or make 
claims, nor did it consider informing the regulatory 
college or professional association of the potential 
misconduct of the professional in question. 


Respiratory Devices 


We noted that the Ministry identified potential con- 
flict of interest when it reviewed vendors’ claims for 
respiratory devices from the 2004/05 fiscal year. 
For example, one review noted that “clinic staff or 
physicians referred clients to the vendor—unless 

a formal contract is entered into with physicians, 
the Program cannot exercise effective control over 
physicians.” It then recommended “urgent action 
be taken with a view of entering into contractual 
agreements with all clinic physicians with par- 
ticular emphasis on conflict of interest.” Another 
review revealed, “some prescribing physicians had 
referred clients to the vendor whose business was 
operated from the same buildings that housed the 
clinics. This would indicate a conflict of interest by 
prescribing physicians. This matter requires urgent 
attention as it similarly affects other physicians and 
vendors.” 

We were informed that, even though the 
Ministry was aware of this problem and indicated 
that “urgent action” and “urgent attention” were 
required, no action has been taken over the past 
few years to address it. The Ministry told us it could 
not investigate the prescribing physicians and 
sleep clinics because the Program does not have 
contractual agreements in place to enable it to do 
so. The Ministry indicated that it would obtain legal 
advice on this issue and pursue the matter with the 
Ministry’s Fraud Programs Branch. 
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In our audit, we observed apparent conflict 
of interest between vendors of CPAP devices and 
prescribing physicians that we believe warranted 
further investigation. These were similar to our 
observations in other device categories. Here are 
two examples: 

@ One vendor submitted more than 5,500 claims 
for CPAP devices, amounting to $4.7 mil- 
lion, since it registered with the Program in 
the 2003/04 fiscal year. We noted that one 
physician prescribed about 94% ($4.4 mil- 
lion) of these claims. This indicates potential 
financial dependence between the vendor and 
physician, and therefore potential conflict of 
interest. In 2005, the Ministry reviewed this 
vendor and noted similar concerns. It also 
found the referring physician had clinics in 
three different municipalities. Clients were 
travelling from these various locations to 
purchase CPAP devices from this one vendor, 
which suggested that the vendor was using 
the clinics to obtain referrals. As part of its 
review, the Ministry sent out confirmation 
letters to the vendor’s clients and half of those 
who responded indicated that they had been 
referred to this vendor by the physician or 
by clinic staff. Ministry staff told us that they 
have taken no action against the vendor or the 
physician. 

@ The same vendor has another location, 
which was registered with the Program in 
the 2005/06 fiscal year. We observed similar 
problems to those described above. This loca- 
tion has submitted about 2,700 claims for 
CPAP devices, amounting to $2.3 million. One 
physician prescribed about 92% ($2.1 mil- 
lion) of these claims. We also noted that, 
in 2008, the Ministry received a complaint 
about the physician directing a client to buy 
a device from a specific vendor. When the 
client refused to do so, the physician threw 
the application form at the client. Despite the 
seriously inappropriate behaviour described 
within, this complaint was never forwarded 


to the appropriate program staff for further 
review or brought to the attention of the 
appropriate regulatory college. The Ministry 
informed us that it typically advises clients 
who have complaints about their physician 
to contact the College of Physicians and Sur- 
geons of Ontario. 


| RECOMMENDATION 6 


To deter potential conflict of interest as well as 

the misuse and abuse of program funding, the 

Ministry of Health and Long-Term Care should: 

e more closely monitor vendor billing patterns 
and, particularly when claims have increased 
dramatically, consider investigating the . 
various parties for evidence of inappropriate 
authorizing or billing practices; 

® terminate agreements with vendors and 
authorizers who breach the Program’s con- 
flict of interest policies; and 

® inform the appropriate regulatory college or 
professional association of any health-care 


professionals whose behaviour or practices 
put the public at risk of harm. 





In 2008, the Ministry received approval to 
develop a new information system to replace 
its current legacy system; the new system is 
expected to be implemented in spring 2011. Sys- 
tem re-development will support the Program by 
enhancing monitoring capacity. The new system 
will also help the Ministry to monitor patterns 
and trends of authorizer and vendor activity. 
The Ministry is proactively working to 
strengthen compliance with program policies 
and procedures. It is reviewing vendor contracts 
and authorizer agreements to establish stricter 
rules on conflict of interest and actions to be 
taken in instances of non-compliance. The 
Ministry will also liaise with the appropriate 
regulatory colleges to determine contacts and 
protocols. y 


RECYCLING AND REFURBISHING 
INITIATIVES 


The Ministry could achieve savings and protect the 
environment by recycling and refurbishing devices 
that clients are no longer using. However, we 

noted that the existing processes did not allow the 
Ministry to maximize the number of recycled and 
refurbished devices, particularly for high-cost items 
such as wheelchairs. 


Power Wheelchairs 


Because of the high cost of power wheelchairs, the 
Ministry established a Central Equipment Pool for 
High Technology Wheelchairs (CEP) in 1996. CEP 
provides clients throughout Ontario with both new 
and recycled power wheelchairs and gives clients 
rebates when they return the equipment to the 
pool. CEP also provides all routine maintenance 
and repair free of charge. Through a competitive 
tendering process in 2007, a vendor was awarded 

a three-year contract to manage and operate CEP 
from March 2007 to February 2010. The vendor has 
guaranteed a recycling rate of 20% in its first year 
of operation and 25% thereafter, with any shortfall 
to be credited to the Ministry. We found, however, 
that the actual recycling rate in the first year was 
8.4%. The rate in the second year was yet to be 
determined at the time of our audit, because the 
year had just ended. We also noted that a refund 
for the shortfall had yet to be made to the Ministry. 
When we brought this issue to the Ministry’s atten- 
tion, we were informed that it would follow up with 
CEP on this matter and obtain a refund if the target 
rate had not been met. 


Manual Wheelchairs 


Since the 2002/03 fiscal year, manual wheelchairs 
have accounted for about 80% of all wheelchair 
claims, with power wheelchairs and power scooters 
accounting for only about 15% and 5%, respect- 
ively. Yet there is currently no recycling initiative in 
place for manual wheelchairs. 
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The Ministry informed us that it had done a 
study in 2003 that proposed to establish regional 
recycling equipment centres for manual wheel- 
chairs to help manage the costs associated with the 
increased demand of a growing and aging popula- 
tion. The study noted: “Introducing equipment 
centres for manual wheelchairs is a wise use of 
health-care resources. Recycling expensive equip- 
ment such as wheelchairs is good for clients, the 
health care budget and the environment. Clients 
and their families have shown strong support for 
recycling.” In addition to the lower environmental 
impact, the Ministry’s study estimated that the 
Ministry could save $11.5 million from 2003/04 to 
2006/07 by recycling manual wheelchairs. Despite 
its significant potential savings and benefits, this 
initiative has not been put in place and there is cur- 
rently no plan to implement any recycling initiative. 
Ministry staff indicated that they have concerns 
about guarantees on the quality and strength of 
recycled parts, the cost of servicing used devices, 
and legal liabilities. 

Our review of other jurisdictions showed that 
provinces such as Alberta and Quebec have manual 
wheelchair recycling initiatives in place. We 
learned in our discussions with them that they had 
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considered some of the same issues around recyc- 





ling, such as potential liabilities and costs, and still 
found that implementation was viable. We noted: 

@ The Alberta wheelchair recycling program has 
been in place for more than 20 years. Alberta 
funds the recycling of both manual and power 
wheelchairs. The program manager told us 
that it is better to recycle manual wheelchairs 
than power wheelchairs because the aver- 
age transaction costs—including cleaning, 
repairing, and refurbishing—are less than 
$400, about one-third of the cost of a new 
manual wheelchair (basic model). Accord- 
ing to the program manager, the wheelchair 
recycling program saves Alberta about $5 mil- 
lion a year. 

® Quebec started a pilot project of wheelchair 
recycling in 2000 that was modelled on 
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Alberta’s program. All devices are recycled 
and distributed directly by accredited rehabili- 
tation centres to ensure the quality of the 
recycled devices. Each centre has its own local 
depot. The program pays the centres to refur- 
bish the wheelchairs. In the 2005/06 fiscal 
year, a program evaluation found that about 
29% of the wheelchairs were recycled, and 
estimated that the average cost of a recycled 
wheelchair was, again, about one-third of a 
new one. Quebec’s recycling program is newer 
than Alberta’s program, but it has still resulted 
in a savings of about $4 million per year 
according to the evaluation report. 
Our review of literature on Quebec’s recycling 
initiative published by the Canadian Association 
of Occupational Therapists in 2003 showed that, 
although program staff at that time indicated that 
there was a lack of resources and no policy in place 
to encourage people to recycle, both occupational 
therapists in the community and users of refur- 
bished wheelchairs reported high satisfaction with 
regard to the efficacy, appearance, safety, durabil- 
ity, and comfort of the recycled devices as well as 
the delivery and follow-up services they received. 
The Quebec Auditor General’s report for the 
2005/06 fiscal year also indicated that the recycling 
program was cost-effective and achieved significant 
savings. 


Incentive to Recycle 
We noted that there are recycling initiatives for 
manual wheelchairs that have been started by vol- 
unteers in Ontario communities through some non- 
profit organizations. The information we obtained 
from such organizations indicates that because 
the Ministry currently does not fund used devices, 
authorizers have no incentive to advise their clients 
to look into buying used or refurbished devices. 
One of the organizations told us that it has a short- 
age of space because of its growing accumulation 
of used devices. This organization also told us that 
it is constantly hearing from people who want to 


donate items, but its limited warehouse space has 
been filled to capacity. It has been giving away 
wheelchairs to other countries to help deal with the 
shortage of space. Ontario taxpayers’ dollars are in 
turn subsidizing health care in other countries. 

Unfortunately, even if clients wanted to get a 
recycled wheelchair, there is little financial incen- 
tive for them to do so under the Program’s current 
funding practices. Clients would have to pay more 
for a recycled wheelchair than they would for a 
new, Program-funded one: the Program-approved 
price for a new basic manual wheelchair is about 
$1,200, of which the client has to pay 25%—about 
$300; if the client wanted to buy a similar used 
manual wheelchair, it would cost about $400, 
which is only one-third the cost of a new wheel- 
chair but still $100 more out of the client’s pocket, 
because used manual wheelchairs are not eligible 
for program funding. 


RECOMMENDATION 7 





To achieve cost savings and protect the environ- 
ment, the Ministry of Health and Long-Term 
Care should consider the feasibility of imple- 
menting a strategy to recycle and refurbish used 
manual wheelchairs based on the experience 

of other jurisdictions that have successfully 
adopted such a strategy. 





The Ministry has noted that some other 


jurisdictions have included recycled manual 
wheelchairs in their programs but has not 
determined that this would be a cost-effective 
approach given the very limited warranty that 
can be provided to refurbished wheelchairs. The 
Ministry will promote the reuse of wheelchairs 
in the context of recycling materials used in 
wheelchair manufacturing. 


RECOVERY OF OVERPAYMENTS 
Deceased Clients 


We noted many instances of an unreasonably long 
time lag between the date of a home oxygen client’s 
death and the date the Ministry’s records were 
updated, which creates a risk of payments being 
continued long after a client is deceased. Since the 
2003/04 fiscal year, the Ministry has recovered 
about $1.2 million from home oxygen vendors 
that had received payments for clients who were 
deceased. However, at the time of our audit, the 
Ministry was still identifying potential recoveries 
that dated back to 2001. Ministry staff informed us 
that these outstanding recoveries had been omitted 
from earlier overpayment reports and that work is 
underway to fix this problem. 

With respect to ostomy grants, the Ministry 
requires clients to complete a renewal form every 
two years to confirm that they still have their 
ostomy (or ostomies). The Ministry also links the 
Program’s database with the Registered Persons 
Database to verify ostomy clients’ health card 
status, and cancels grants automatically if the 
renewal form, cheque, or direct deposit is returned 


as undeliverable. These steps have been successful - 


in reducing the number of payments being made 
to deceased persons, but there is still a time lag 
between the date of a person’s death and the date 
the Ministry updates its records. The last report of 
these overpayments was generated in June 2008, 
but the executors of the estates of the deceased 
clients have not all been contacted. 


Duplicate Funding 


Under the Program’s general eligibility rules, 
individuals who are eligible for funding for their 
devices from the Workplace Safety and Insurance 
Board (WSIB) or the federal Department of Vet- 
erans’ Affairs (DVA) are not eligible for program 
funding. Applicants for program funding are 
required to declare on their application that they 
are not eligible for funding from the WSIB or the 
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DVA, but the Ministry does not obtain independent 
verification of this information. We identified the 
same issue in our 2001 audit. 

In 2004, the Ministry’s Fraud Programs Branch 
also identified this issue. The Branch stated that the 
Ministry should not be compensating clients unless 
they have maximized benefits from other sources. 
It pointed out that, without direct data-links to the 
WSIB and the DVA, there is a risk of the Ministry 
funding devices for individuals who are entitled 
to compensation through the WSIB and the DVA. 
There is also a risk of unscrupulous vendors billing 
more than one agency for the same device. The Min- 
istry’s Fraud Programs Branch recommended that 
the Program continue to negotiate an information- 
exchange agreement with the WSIB and initiate an 
agreement with the DVA to identify ways in which 
the risk of double billing could arise. 

During our current audit, we noted that the Min- 
istry still had no direct access to the WSIB and the 
DVA databases. An information cross-check process 
with the WSIB was discontinued in 1998 and has 
not been re-instituted, and similar arrangements 
with the DVA were never put into place. The Min- 
istry entered into an agreement with the WSIB in 
1999 to recover duplicate funding for hearing aids. 
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So far it has recovered duplicate funding of about 
$110,000 for hearing aids since 2006, but no similar 
recovery has been made in other device categories. 





In our review of program and WSIB claims data 
since 2002/03, we noted cases where the Program 
and the WSIB provided funding to the same person 
for the same category of device around the same 
time. The Ministry has not yet followed up on these 
cases, which involve funding of $760,000. 


~ RECOMMENDATION 8 


To ensure that Assistive Devices Program grants 
are administered economically, the Ministry 

of Health and Long-Term Care should recover 
overpayments on a timelier basis and expedite 
the recovery of overpayments made since 2005. 
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To ensure that funding for devices is not 
duplicated at taxpayers’ expense, the Ministry of 
Health and Long-Term Care should re-institute 
an information-exchange agreement with the 
Workplace Safety and Insurance Board and initi- 
ate an agreement with Department of Veterans’ — 
Affairs as has been recommended by the Min- 
istry’s Fraud Programs Branch. 

To date, the Ministry has recovered all overpay- 


ments that it is aware of; reports are generated 
weekly and the Ministry will continue to recover 





overpayments. 

The Program has an agreement in place 
with the Workplace Safety and Insurance Board 
(WSIB) to recover duplicate payments for 
hearing devices that are required as a result of 
a workplace injury. The Ministry is discussing 
with the WSIB the potential for other device cat- 
egories to be included in the agreement, and is 
also discussing with Veterans Affairs Canada the 
potential for an efficient exchange of informa- 
tion to identify duplicate payments. 


REGISTRATION OF AUTHORIZERS 


In most cases, individuals applying for program 
funding are required to be assessed by Program- 
registered medical authorizers and must purchase 
their devices from Program-registered vendors (see 
Figure 3). Our sample testing identified the follow- 
ing concerns with authorizer registration. 

We reviewed a sample of authorizers’ files to 
assess whether they had met the criteria to register 
with the Program. Some of the documents were 
missing from files, so we could not determine 
whether all the registration requirements (see 
Figure 3) had been met. The missing documents 
included proof of good standing with the appropri- 
ate regulatory colleges and proof of completion of a 
required course or workshop. 


Every three years, the Program requires author- 
izers to submit an Information Update Form. 
Authorizers must submit their updated contact and 
professional information to maintain their active 
status with the Program and to obtain new Author- 
izer Cards, which are displayed to help clients 
confirm that an authorizer is in fact registered with 
the Program. 

The process of renewing authorizers’ status 
was not monitored appropriately. We noted the 
following: 

e The Ministry did not follow up with author- 
izers who had not returned their Information 
Update Forms. Only after we found that the 
forms were missing in the files did the Min- 
istry send reminder letters. 

@ One authorizer’s former employer wrote to 
the Program in 2005, asking why they had 
recently received a letter and a new Author- 
izer Card expiring in May 2008 for someone 
who was no longer employed with them and 
who had been out of the province since 2002. 
This suggests that the Program was issuing 
new Authorizer Cards without verifying 
authorizers’ information. 

To verify authorizers’ status, we contacted five 
professional colleges that regulate authorizers. 

We noted instances where the Ministry did not 
promptly update authorizers’ status. For example: 

@ Some authorizers were not in good standing 
with their colleges, but the Ministry did not 
deactivate their registration status until five to 
ten years later. 

@ Some authorizers’ had active status with the 
Program even though their colleges’ records 
showed their membership had been deacti- 
vated in 2006 or 2007. 

@ Some authorizers continued to authorize 
devices when they were not in good standing 
with their colleges. The total value of claims 
related to these devices was about $400,000. 

In 2004, a report by the Ministry’s Fraud Pro- 
grams Branch recommended that the Program 
increase its due diligence on the licensing status 
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of authorizers by generating links with the profes- } 
e follow up on those authorizers who do not 


submit the required Information Update 
Forms. 


sional colleges to help with ongoing monitoring. 
However, we noted that the Program still has not 
developed direct data-links with the colleges to 


them. Thisincessestherskefpogam ey 
them. This increases the risk of program abuse by 


authorizers who have been suspended or who are The Ministry is reviewing the June 2009 amend- 
no longer practising. Ministry staff informed us that ment to the Regulated Health Professions Act to 
the Program would continue to look into solutions determine if the legislation allows sufficient 
with the colleges; the Program has also set up a access to information to conduct ongoing 
committee to identify strategies for improving its monitoring to ensure that authorizers are in 
management of authorizers. good standing with their regulatory bodies and 


whether additional channels will need to be 
RECOMMENDATION 9 developed. 


: ies : : The Program terminates the status of 
To lower the risk of assistive devices being ‘ : 
authorizers who do not return Information 


d for funding by authori h 
Sees te 2 MOUIZETS Wil0 ate Update Forms within the specified timeframe. 


not properly registered with the Program, the 

Ministry of Health and Long-Term Care should: 

@ generate links with the professional colleges 
to enable ongoing monitoring of authorizers’ 
status; and 


These authorizers are required to re-register 
with the Program to become active again. 
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Section 


Background 


Ontario has about 14,800 bridges. Approximately 
12,000 of these are located in municipalities and 
are their responsibility. The remaining approxi- 
mately 2,800 bridges are located within the prov- 
incial highway system and are the responsibility of 
the province. More than 70% of provincial bridges 
were built between 1950 and 1980, which gives the 
province’s bridge infrastructure an average age of 
about 40 years. 

In the past, bridges were expected to last about 
60 years; however, current technology and design 
allow the bridges that are built today to last longer. 
High traffic volume, heavy trucks, and freeze/thaw 
cycles along with exposure to salt used for winter 
maintenance all reduce a bridge’s lifespan. Regular 
maintenance, repair, and rehabilitation can largely 
offset the impact of these factors on a bridge’s 
lifespan. 

Responsibility for the safety and maintenance of 
provincial bridges is set out in the Public Transpor- 
tation and Highway Improvement Act (Act). The Act 
requires that all provincial and municipal bridges 
be inspected every two years under the direction of 
a professional engineer using the Ministry’s Ontario 
Structure Inspection Manual (Inspection Manual). 
The Inspection Manual requires these biennial 
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inspections to be a “close-up” visual assessment 

of each element of a bridge as well as its material 
defects, performance deficiencies, and maintenance 
and rehabilitation needs. 

The Ministry of Transportation (Ministry) is 
responsible for provincial bridges, and munici- 
palities are responsible for the bridges in their 
jurisdictions. 





Audit Objective and Scope 


The objective of our audit was to assess whether the 
Ministry: 

e has effective systems and procedures in place 
to ensure that the bridges within its highway 
system are safe and in good repair; and 

@ conducts bridge inspections and the required 
maintenance, repair, rehabilitation, and 
replacement work on a timely basis and with 
due regard for economy. 

Our audit included visits to the Ministry’s head 
office and three of its five regional offices. We inter- 
viewed staff, examined documentation, reviewed 
the results of bridge inspections, and researched 
bridge management practices followed in other 
jurisdictions. We also accompanied ministry staff 
on bridge inspections to gain an understanding 


of the inspection process and the challenges that 
inspectors face. 

In the course of our audit, we engaged a struc- 
tural engineering expert from another province to 
review the Ministry’s bridge inspection standards 
and practices and to help us interpret the results 
of the bridge inspections. To obtain an independ- 
ent assessment of the condition of four provincial 
bridges, we hired an engineering firm to re-inspect 
them using the Ministry’s Inspection Manual. 

In 2004, we conducted an audit of the mainten- 
ance of the provincial highway system, and one 
of our recommendations specifically related to 
the Ministry’s bridge inspection processes. Our 
current audit included a follow-up on the status 
of actions taken on this recommendation. The 
Ministry’s Internal Audit Services had also issued 
a report in September 2005 on bridge inspection 
processes covering the period from January 1, 
2001, to December 31, 2004. The report, as well as 
its follow-up in 2007, resulted in improvements to 
areas such as the timeliness and accountability of 
the inspection process. As well, they were useful in 
determining the scope and extent of our audit. 


MUNICIPAL BRIDGES 


Although municipalities must inspect their own 
bridges in accordance with the Inspection Manual, 
there is no legislation that provides any provincial 
ministry with the authority to oversee municipal- 
ities’ compliance with this requirement. Given that 
the majority of the bridges are within municipal 
boundaries, and recognizing that the province 

still has an overall responsibility for the legislation 
governing bridge safety, we conducted a survey of 
about 130 Ontario municipalities, to which almost 
60% responded. Our objective was to obtain infor- 
mation on how municipalities kept track of bridge 
inventories, what systems they used for comply- 
ing with and reporting on required inspections, 
and what their perspectives were on the current 
operating and funding arrangements. We met with 
representatives from 10 large municipalities to 
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further discuss their survey responses, and also met 
with representatives from the Association of Muni- 
cipalities of Ontario and the Ontario Good Roads 
Association. The results of our survey and discus- 
sions are included in this report in the Municipal 


Bridges section. 





PROVINCIAL BRIDGES 


The structural engineering expert we engaged 
advised us that the Ministry of Transportation (Min- 
istry) had established comprehensive standards 

for bridge inspection in the Ontario Structural 
Inspection Manual (Inspection Manual), and if the 
standards are followed, the required inspection 
procedures effectively enable structural deficiencies 
to be identified. The Ministry’s standards had been 
adopted for use by a number of other Canadian 
jurisdictions. The Ministry is also conducting bridge 
inspections on a biennial basis as required. 

The main safety risks related to bridge infra- 
structure are accidents, such as those caused by 
concrete falling, or parts of a bridge structure fail- 
ing to perform their intended function of providing 
adequate protection to the vehicles travelling on 
the structure. We noted a number of areas where 
improvements to the Ministry’s inspection and 
maintenance processes would help minimize these 
safety risks and ensure that these bridges for which 
the province is responsible remain safe, especially 
given the aging infrastructure. Our specific obser- 
vations are as follows: 

e According to the Ministry’s assessment, more 
than 180 or 7% of provincial bridges were in 
poor condition, defined as requiring repair 
or rehabilitation work within one year of the 
bridge inspection. We found that, despite 
their being in most need of repair or rehabili- 
tation, over one-third of these bridges were 
not included in the Ministry’s capital work 
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plan for the upcoming year. The Ministry 
indicated that it takes a corridor management 
approach to prioritizing such work that con- 
siders factors other than the rated condition 
of the bridge. As well, the Ministry stated that 
any critical safety issues would be flagged 
during inspections and remedial work carried 
out immediately. However, our engineering 
adviser indicated that, often, a distinction was 
not made in the inspection report between 
deficiencies that posed a safety risk and those 
that did not. 
The Ministry had not ensured that informa- 
tion on critical elements within each bridge 
was accurate and that all elements were 
accounted for. The state of these elements is 
the key to determining a bridge’s overall con- 
dition and estimating its rehabilitation costs. 
In addition, the Ministry’s database of bridge 
inventory—the Bridge Management System— 
did not have information on the rehabilitation 
history for almost one-third of the bridges that 
were 40 years or older. Although this informa- 
tion might be available in a region’s paper files 
or local database, the Ministry’s prioritiza- 
tion, cost estimates, and timelines for bridge 
rehabilitation work would be enhanced if this 
information was made readily available. 
The Inspection Manual requires a detailed 
visual “close-up” inspection of each bridge 
element. Normally, this requires the closure 
of lanes and road shoulders to traffic. For 
example, without closing a lane, close-up 
inspection of the critical elements of certain 
bridges on Highway 401 in the Greater 
Toronto Area would not be possible, yet there 
have been no such lane closures for the past 
three years. 
We found several weaknesses regarding the 
process for ongoing oversight of inspections. 
For example: 
e The Inspection Manual stipulates that an. 
inspector needs to spend at least two to 
three hours at a typical bridge site. How- 


ever, inspectors were often conducting five 
or more inspections a day. For example, in 
the rounds of inspections between 2006 
and 2008, we noted that 10 or more bridges 
were inspected by a single inspector in one 
day on 36 separate occasions. 

e Asignificant change in the rating of a 
bridge’s condition between inspections 
requires explanation and, potentially, a 
re-inspection. We noted that the latest 
inspection results showed an improvement 
in the overall condition rating of over 300 
bridges, even though little or no rehabilita- 
tion work had been done on these bridges 
since the last inspection. In other instances, 
the overall rating did not change at all 
between inspections and reports from the 
previous inspections were carried forward 
without any changes. Although in many 
cases there were photographs on file to 
indicate that an inspection had been done, 
when there are no changes whatsoever 
from the previous inspection, the adequacy 
of such inspections should be followed up 
on, especially on older bridges, because, 
typically, a bridge’s elements deteriorate as 
the bridge ages. 

e We noted that regions tended not to complete 
many of the maintenance recommendations 
resulting from biennial bridge inspections. In 
two of the three regions that we visited, only 
about one-third of the recommended main- 
tenance work was actually completed, and the 
third region did not track this work at all. 

With respect to the procurement of major pro- 

jects for bridge design and construction, we noted 
that the Ministry generally followed a competitive 
selection process. However, in many of the con- 
tracts for design services and construction oversight 
consulting that we examined, there were changes 
to the scope of work that resulted in a final price of 
at least 50% more than the original contract price. 


MUNICIPAL BRIDGES 


To ensure the safety of municipal bridges, muni- 
cipalities are also required to perform biennial 
inspections in accordance with the Inspection 
Manual. However, there is currently no legisla- 
tion that requires or even enables the Ministry of 
Transportation or any other provincial ministry 

to oversee municipalities’ compliance with this 
requirement. There is no central database on the 
number of municipal bridges and the overall condi- 
tion of these bridges. 

Our survey of municipalities indicated that the 
average age of municipal bridges was generally 
older than provincial bridges—about 43 years. 
However, it was not possible to get a precise picture 
on the overall condition of municipal bridges or to 
make accurate comparisons between municipal and 
provincial bridges because there were many differ- 
ent systems used by municipalities to classify and 
determine the condition of their bridges. 

Nevertheless, the majority of municipalities 
(85%) that responded to our survey indicated that 
they had a backlog of rehabilitation work. Large 
and growing communities generally did not have 
significant backlogs because their infrastructure 
was newer, but municipalities with a large number 
of bridges relative to their population and revenue 
base had more difficulty funding the rehabilitation 
of bridge infrastructure and therefore had more 
significant backlogs. 

In recent years, the province has provided 
municipalities with one-time funding for municipal 
capital projects. The decision to make these funds 
available was often made on the basis of demo- 
graphic information rather than need, the funds 
were paid close to the end of the province’s fiscal 
year, and many municipalities were not able to 
properly plan and spend the money. For instance, 

a significant portion of the funds provided in 2008 
remained unspent one year later. Municipalities told 
us that requirements for better asset-management 
practices supported by more sustainable provincial 
funding are needed to ensure safety and maximize 
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the lifespan of municipal bridges. A provincial— 
municipal working group is currently examining 
these issues. 


Ensuring that Ontario’s bridges are safe facili- 
tates the continuous movement of people and 
goods, supporting the provincial economy. The 
Ministry values the Auditor General’s observa- 
tions and recommendations and is committed to 
taking action. 

Ontario is proud of its reputation as a North 
American leader in bridge safety. Overall, the 
Ministry’s procedures to inspect and monitor 
bridge conditions are comprehesive and 
adequate to ensure that bridges in Ontario are 
safe. Every two years, Ontario inspects all 2,800 
of our provincially owned bridges. Since 2005, 
the government has increased infrastructure 
spending to maintain, rehabilitate, and replace 
bridges. Commitments include an increase of 
50% or $450 million over a five-year period to 
rehabilitate approximately 150 bridges through- 
out the province by 2013. Since 2004/05, the 
Ministry has built 75 new bridges and rehabili- 
tated 388 existing structures. 


Detailed Observations 





BRIDGE INSPECTION PROCESS 


The main objectives of an inspection are to ensure 
that a bridge is in a safe condition; to identify 

any maintenance, repair, and rehabilitation that 
needs to be done; and to provide a basis for plan- 
ning and funding any required maintenance and 
rehabilitation. 

Two of the Ministry’s five regional offices mainly 
use in-house engineers to conduct bridge inspec- 
tions. The other three regional offices outsource 
bridge inspections to private engineering firms. 


Chapter 3 * VFM Section 3.02 





| 84 2009 Annual Report of the Office of the Auditor General of Ontario 


N 
S| 
© 
S 
o | 
= 
o | 
@ | 
” | 
= 
i 
=> | 
e 
o | 
he 
ao 
= i 
2. 
& 
S 
Oo 





The time required to inspect a bridge varies with 
the bridge’s size and design, but the average bridge 
typically takes about two to three hours. If during 
a visual inspection the inspector feels that more 
detailed information is needed, he or she can 
request specialized inspections, such as a deck 
condition survey or fatigue investigation. Also, the 
inspector is to notify the region immediately about 
any critical structural defects or deficiencies and 
any other unsafe condition discovered in the field, 
so that appropriate action can be taken. 

To manage the inventory of provincial bridges, 
the Ministry uses a database called the Bridge 
Management System (System). It contains physical 
and historical information for each bridge, such 
as the length, number of spans, the area of each 
bridge element, the results from each inspection, 
and the condition each element is assessed to be in, 
from poor to excellent. Using this information, the 
System calculates a single value called the Bridge 
Condition Index, a measure of a bridge’s overall 
structural condition and its remaining economic 
value expressed in a number between zero and 
100. Bridges with a Bridge Condition Index of 70 
or above are generally considered to be in good 
condition. 

In 2006, the Ministry developed a Bridge 
Priority Tool to help prioritize major yearly bridge 
repairs and rehabilitation, and standardize the dif- 
ferent priority-setting methodologies used by vari- 
ous regions. The development of the tool was part 
of a commitment that the Ministry had made in 
response to a recommendation in our 2004 Annual 
Report to develop a framework to better manage 
its assets and to set priorities for sound investment 
decisions. The tool calculates a Priority Index for 
each bridge by modifying the Bridge Condition 
Index value after examining the condition of five 
critical bridge elements: the deck top, deck soffit, 
barrier wall, expansion joints, and concrete/steel 
beams. (Figure 1 is a photograph of two bridges 
with a number of these bridge elements labelled.) 
Each of these critical elements is assigned a “now 
need threshold” (the percentage of the element that 





Figure 1: Elements of a Bridge 
Source: Ministry of Transportation 
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is in poor condition and should be repaired within a 
year) and a weight (the importance of the element 
in relation to the entire structure). Bridges with a 
Priority Index of less than 70 are considered to be 
on the zero-to-five-year rehabilitation list; bridges 
with a Priority Index of less than 60 are considered 
to be a “now need,” which means they should be 
rehabilitated during the next construction season. 

The Bridge Priority Tool also estimates the cost 
of future bridge repair and rehabilitation needs. 
Each of the province’s regions are supposed to use 
these rankings to develop a five-year capital work 
plan for repair and rehabilitation work. These 
regional plans become part of the larger provincial 
work plan. 

We engaged a structural engineering expert 
to perform an independent assessment of the 


Ministry’s bridge inspection standards and practi- 
ces. Our expert’s opinion was that the Inspection 
Manual is comprehensive and, if its inspection 
procedures are followed, an effective means of 
identifying significant structural deficiencies in a 
bridge. Our expert also confirmed that the Min- 
istry’s methodology for prioritizing bridge repair 
and rehabilitation work is reasonable. We also 
noted that several other Canadian jurisdictions 
have adopted the Ministry’s Inspection Manual. 


CONDITION OF PROVINCIAL BRIDGES 


According to the latest Priority Index that was avail- 
able to us in June 2009, which included inspection 
results for up to the end of 2008, approximately 
76% or 2,150 of the 2,800 provincial bridges were 
in good condition, 17% or 471 bridges were in fair 
condition but require work within five years, and 
7% or 185 were in poor condition and need repair 
within one year. 

Using its Bridge Priority Tool, the Ministry esti- 
mates that the cost of repairing and rehabilitating 
bridges in fair or poor condition over the next five 
years will be approximately $2.2 billion. Yet the 
actual funds committed to the Ministry’s budget for 
all bridge work over the next five years is $1.4 bil- 
lion, a shortfall of $800 million. The Ministry has 
identified another spike in the need for major 
capital work over the next six to ten years as bridges 
continue to age: 70% of the provincial bridges 
were built between 1950 and 1980, and these older 
bridges have an average lifespan of 60 years. The 
Ministry has projected that an extra $4.2 billion 
will be needed to repair these bridges. 

In light of the expected shortfall, existing funds 
should be spent on bridges with the most urgent 
need for repairs. However, our review found that 
about 60% of bridges rated in poor or fair condi- 
tion were not on the Ministry’s five-year capital 
work plan. Specifically, of the approximately 185 
bridges that were in the “now need” category in 
2008/09, 71 bridges with repair and rehabilitation 
costs estimated to be $190 million were not on the 
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Ministry’s work plans for the following year. Ideally, 
bridges identified as “now need” would have been 
considered for repair and rehabilitation several 
years before they reach this condition, as sufficient 
lead-time is required to arrange for the necessary 
procurement work and ensure that safety concerns 
do not develop in bridges while they await sched- 
uled repairs. 

In response to our observation regarding the 
large population of bridges in fair and poor condi- 
tion that was not on the work plans, the Ministry 
indicated that the Bridge Condition Index was a 
tool that measures the relative overall condition of 
a bridge and is not necessarily a measure of bridge 
safety. Any urgent safety issues would be flagged 
during inspections and remedial work would be 
carried out immediately. 

Rather, the tool was being used primarily 
to assist with planning and prioritizing bridge 
rehabilitation work. In that regard, the Ministry 
indicated that, in response to recommendations 
from our 2004 audit, it has been taking more of a 
corridor management approach to its activities. 
Consequently, the condition rating of a bridge was 
not the only factor considered in prioritizing the 
timing of bridge rehabilitation work. In making the 
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decisions, ministry staff applied their experience 
and judgment and took into account other safety 
and economic factors such as the role of a bridge, 
the kind of traffic it handles now and in the future 
and the cost savings expected through co-ordinat- 
ing bridge work with other highway work. 

The application of the sound asset-management 
principle is indeed key to setting priorities for bridge 
rehabilitation work and making sound capital 
investment decisions. However, we noted that the 
process was still in transition and the Ministry had 
not established formal guidelines for proper appli- 
cation of the principle. As such, the rationale and 
support for such decisions were often neither docu- 
mented nor kept in the Bridge Management System. 
Better analysis and documentation of the safety and 
economic considerations are needed, particularly in 
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cases where the rehabilitation decisions were incon- 
sistent with the condition rating of the bridge. 

Furthermore, given the large proportion of “now 
need” projects that were not on the Ministry’s five- 
year work plan and the lack of documented ration- 
ale for their exclusion, we believe more attention 
needs to be paid to bridges identified as “now need” 
in the Ministry’s priority-setting process. 


SAFETY OF PROVINCIAL BRIDGES 


The primary objective of a bridge inspection is to 
provide assurance that the bridges inspected are in 
safe condition. In this regard, we engaged the ser- 
vice of an engineering firm (firm) to re-inspect four 
of the provincial bridges so that we could obtain an 
independent assessment of risks and the condition 
of those bridges. 

According to the engineering firm, there are two 
main risks with respect to the poor condition of 
bridge infrastructure. One is the risk of accidents, 
which can be caused by objects falling and hitting 
traffic or leaving obstacles on the roadway below; 
or parts of the bridge structure failing to perform 
their intended function of providing adequate pro- 
tection to the vehicles travelling on the structure. 
The other risk is related to the deterioration of 
property that could result in expensive repairs and 
rehabilitation over the long run. 

The engineering firm concluded that, overall, the 
Ministry’s inspections were carried out in conform- 
ity with the intent of the Inspection Manual and that 
most critical features were recognized and pointed 
out. However, there were several notable exceptions 
where the risk of accidents could be serious: 

e@ For all four of the bridges, the Ministry had 
not thoroughly inspected some significant 
bridge elements, including the underside, side 
faces, and piers because of traffic (see also the 
section Gaining Access to Bridges for Inspec- 
tion). This could impede the effectiveness of 
the inspection and pose a serious risk (such as 
of concrete falling onto traffic). 


@ Our firm indicated that some of the features 
that can be seen or felt (for example, vibra- 
tions) can best be described in narrative form. 
Each critical bridge element on the Ministry’s 
inspection form is assigned a value from poor 
to excellent. Inspectors are expected to inter- 
pret what they see or hear and indicate on 
the inspection form the recommended timing 
of bridge maintenance work—from urgent 
to 10 years—where required. However, the 
inspectors’ assessment does not distinguish 
between deficiencies that posed a risk to 
safety versus deficiencies relating to loss of 
value as measured by the high cost of repairs, 
given the potential impact of the deficiencies 
identified. Our adviser was of the opinion 
that such information should be clearly com- 
municated in the inspection report to provide 
assurance about a bridge’s safety and enable 
better planning of bridge work. For example, 
a ministry inspection report had called for 
repair of defects on some bridge surfaces 
within two years. However, there was no dis- 
tinction made between surfaces and whether 
or not delaminated concrete could fall into 
Highway 401 traffic—an important piece of 
information for assessing the urgency of the 
repair work. 

e Two of the four re-inspected bridges were 
rated in fair or poor condition and were there- 
fore supposed to be rehabilitated within five 
years. The bridge that was in poor condition, 
in particular, was intended to be replaced 
completely in two or three years because of its 
deteriorated state. Neither of the two bridges, 
however, was on the Ministry’s five-year cap- 
ital work plan. 


RECOMMENDATION 1 


To ensure that appropriate and timely action is 
taken on bridges requiring repair and rehabilita- 
tion work, the Ministry of Transportation should: 


@ strengthen its risk-assessment and priority- 
setting process, with particular considera- 
tion given to bridges identified as being in 
poor condition, so that any urgently required 
work is given first priority; and 

@ ensure that government decision-makers 
receive the information they require to 
adequately assess both safety and economic 
risks in order to prioritize the capital needs 
of Ontario’s aging provincial bridges. 


Bridge repairs required to protect the safety of 
the travelling public are performed immediately. 
As shown by its long safety record, the Ministry 
has effective processes in place to address 
short-term urgent repair needs and long-term 
rehabilitation requirements. To build on the 
Auditor General’s suggestion to further enhance 
and strengthen our risk-assessment process, 

we are implementing mandatory detailed 
documentation of bridge safety issues when 
identified by inspectors, clearer identification of 
potential safety risks, and recording of all bridge 
maintenance work that is completed. These 
enhancements will ensure that the Ministry 
further identifies bridges in need of additional 
repair and rehabilitation. : 

Backed by comprehensive inspection reports 
and engineering expertise, we have a multi-year 
work plan for bridge repair and rehabilitation. 
This plan carefully evaluates both bridge safety 
and economic risks to ensure the proper timing 
and location for capital improvements. New 
highway-infrastructure-management software 
will integrate a broader range of bridge condi- 
tion and economic data to effectively prioritize 
bridge repair needs. The Ministry is also imple- 
menting even more rigorous criteria to enhance 
documentation of bridge repair work being 
undertaken or deferred. 
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BRIDGE INVENTORY 


Effective bridge inspection requires complete 

and accurate information on the 2,800 provincial 
bridges and their individual elements. As indicated 
earlier, the information in the Ministry’s bridge 
inventory is recorded on the Ontario Bridge Man- 
agement System database, from which the Bridge 
Condition Index and the Priority Index are derived. 
These indices provide information about a bridge’s 
overall structural condition and form the basis for 
prioritizing bridge repair and rehabilitation. 

The Ministry’s Inspection Manual lists over 
50 elements that may be found in a bridge. Each 
bridge typically has at least 20 of these elements, 
and usually many more. Some examples of bridge 
elements are the deck-top, soffit (underbelly of the 
bridge), columns, and railing systems. During an 
inspection, the inspector is required to check if the 
bridge’s structure and elements match its design 
drawings and the information in the database. If 
there have been any changes, the inspector must 
update the information in the database. 

We found that the Ministry had adequate proced- 
ures in place for ensuring that it identifies all provin- 
cial bridges for which it has responsibility. However, 
the Ministry has not ensured the completeness 
and accuracy of its information on the individual 
elements that comprise each bridge. The Ministry 
cannot be precise in its rating of the overall condi- 
tion of a bridge if the inspection does not assess the 
condition of each of the bridge’s elements. 

As part of its quality-assurance review in 2006 
and 2007, the Ministry’s head office re-inspected 
a sample of bridges that had been looked at by 
in-house inspectors or outside consultants in the 
various regions. For about 75% of the re-inspected 
bridges, at least one error was found in the way the 
information on bridge elements was recorded in 
the Bridge Management System. Examples of errors 
included missing elements and recording the wrong 
quantities or dimensions. About 40% of the errors 
found were considered to have had a significant 
impact on the condition in which the bridge was 
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rated and the resulting time frame for its repair or ‘ ; : : 
Pig oe, inspections, a formal, rigorous oversight process 
rehabilitation. : aie : 
: wt ; has been implemented so that critical bridge 
At the completion of our audit in spring 2009, P ere 
We. information is kept current and recorded. 
the Ministry had not yet fully addressed the results : : 
; : The large amount of data included in the 
of its quality-assurance review and some of the ‘ 
; Bridge Management System must be correct for 
Bridge Management System information remained : ; ; Sa ; 
Wee ’ it to be effectively used in decision-making. We 
inaccurate. We also found significant differences f } , 
are also acting on the Auditor General’s sugges- 
between what was recorded in design drawings and : 
tion to review our data for completeness, mak- 
in the Bridge Management System. For example, the i see 
; ing additions and updates where necessary. We 
dimensions of deck-tops in design drawings did not ; é 
expect this review to be complete by late 2010. 
match what was recorded in the system for about 


25% of the bridges in our sample. For more than 

600 bridges, the dimensions recorded for deck-tops 

differed between different tables within the Bridge GAINING ACCESS TO BRIDGES FOR 
Management System. For approximately 200 of INSPECTION 

these bridges, the difference was greater than 20%, 

The Inspection Manual states that each element of 


which would have a significant impact on the esti- } ; ; : ‘ : 
a bridge is to be inspected in a systematic fashion. 


mated cost of their repair and rehabilitation. 


This means that an inspector is required to record 





x | RECOMMENDATION 2 observations, make sketches where appropriate, 

o and take photographs that clearly show the struc- 

s To better ensure that the results of bridge ture and any defects found within it. 

3 | inspections are accurately recorded and to bet- The Inspection Manual also calls for detailed 

e | ter prioritize and estimate the cost of bridge visual inspections of bridges to be performed regu- 

a repair and rehabilitation, the Ministry of Trans- larly. A detailed visual inspection is an element- 

S| portation should: by-element, “close-up” assessment of a structure’s 

s @ more closely monitor inspectors’ compliance material defects, performance deficiencies, and 

ms with the Bridge Inspection Manual so that maintenance needs. The Manual states that these 
critical bridge information is accurately inspections should be conducted within arm’s 
updated; and length of the element, possibly involving tapping 

e act on findings from its quality-assurance with a hammer or taking measurements by hand. In 

review and ensure the completeness and some cases (such as on structures that are generally 
accuracy of information kept in the Ontario in good condition) it may be possible to inspect 
Bridge Management System. a portion of a bridge up close and then estimate 


the condition of the remaining, inaccessible parts 





through extrapolation. 
Thorough bridge inspections often require that 


The Ministry agrees with the Auditor General 


: : : lanes and road-shoulders be closed. Inspectors are 
on the importance of inspector oversight. To ne 
ek responsible for obtaining the required approval for 
ensure that this continues, a mandatory training 


program is provided to all bridge inspectors, 
clearly identifying specific roles, responsibil- 


lane and shoulder closures through the Ministry’s 
traffic department. However, we found that in 


a : : the past three years there have been only a few 
ities, and requirements. Acting on the Auditor 

; ‘ : shoulder closures and no requests for lane closures 
General’s suggestion to further enhance bridge ; 
in the central region that encompasses the Greater 


Toronto Area. When we questioned this, we were 
informed that it was virtually impossible to get 
approval for these closures in the Greater Toronto 
Area. In other parts of the province, closures are 
possible but were also not widely used as the dis- 
ruption and expense is believed to be unwarranted. 


In a 2006 training workshop, ministry inspectors 


and external consultants both commented that 
their work was challenged by not being able to gain 
adequate access to perform thorough, close-up 
inspections of large bridges. This issue is particu- 
larly serious in the Greater Toronto Area, where 
there are over 660 bridges on the 400 series of 
highways and some of them span up to 16 lanes of 
traffic. Half of these bridges are 40 years or older. 

Having only limited access to bridges means 
inspectors may be forced to leave some elements 
uninspected, or to estimate the condition of some 
elements from afar, which increases the risk of 
inaccurately assessing their condition. Proper 
inspections of significant elements such as the 
soffit, beams/girders, or bearings may not even be 
possible. 

The firm we engaged to re-inspect four of the 
provincial bridges confirmed that the lack of access 
represented a significant hindrance to the inspec- 
tion because some surfaces could not be touched 
and some could only be photographed from afar 
or not at all. Some of these surfaces represent 
important sources of risk. For example, surfaces 
facing traffic on Highway 401 could not be probed 
for delamination, which on all concrete soffits will 
eventually lead to pieces spalling and falling off. 

External consultants at the 2006 training 
workshop suggested that they work with ministry 
engineers to identify any required lane closures 
and that these closures be mandated in their agree- 
ments with the Ministry. The consultants added 
that the number of lanes to be closed should be 
specified when the work was tendered, to ensure 
that all consultants are bidding on the same scope 
of work. The reasoning behind this request is that 
if lane closures are optional, consultants bidding 
on the work might be tempted to omit the cost of 
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the closures from their tender. The Ministry has not 
acted on the consultants’ request. 


~ RECOMMENDATION 3 


To ensure that inspections are carried out in 
accordance with legislation, the Ministry of 
Transportation should: 

@ arrange for the closure of lanes and shoul- 
ders whenever these are required to ensure 
that an adequate bridge inspection can be 
carried out; . 

@ if closure of lanes and shoulders is not 
always possible for every bridge inspection, 
consider a risk-based approach that takes 
into consideration factors such as the age 
of the bridge and the feasibility of rotating 
inspections. Off-peak closures such as at 
night or on weekends also warrant more 
consideration to facilitate bridge inspection; 
and 

e consider specifying lane and shoulder clos- 
ures when tenders are issued for inspections 
to be done by external consultants. 





Ontario has some of the busiest highways in 
North America, making lane and shoulder clos- 
ures very challenging. To build on the Auditor 


General’s suggestions, we are acting on the 
suggestion to implement a risk-based approach 
to bridge inspection to ensure that lane closures 
occur on critical bridges; scheduling 75 traffic 
lane/shoulder closures this year in the Greater 
Toronto Area; and conducting a thorough review 
of all bridges across the province, starting with 
older bridges, where lane closures may be 
needed for effective bridge inspection. Lane clos- 
ures will be mandatory for future inspections at 
these locations. In all cases where lane or shoul- 
der closures are needed, we will work with our 
contractors to minimize impact to traffic without 
compromising the inspection process. 
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INSPECTION OVERSIGHT 


Since its quality assurance review in 2006, the 
Ministry has carried out periodic re-inspections of 
bridges to check whether previous inspections were 
done in accordance with its Inspection Manual. 
The Ministry also conducts training workshops for 
ministry inspectors and external consultants that 
are aimed at improving the quality and consistency 
of bridge inspections. 

These are good initiatives. Nevertheless, we 
identified a number of areas that we believe 
indicate a need for better oversight of bridge 
inspections. 


Time Spent on Inspections 


While the time required for a bridge inspection var- 
ies according to the type and design of the bridge, 
the Inspection Manual states that an inspector 
should plan to spend at least two to three hours at 
a typical bridge site to adequately assess the condi- 
tion of all elements. Larger bridges take longer to 
inspect. 

We noted that, on average, inspectors conducted 
three to five inspections in a single day. In the 
round of inspections done between 2006 and 2008, 
we noted that there were 36 instances where 10 
or more bridges had been inspected by a single 
inspector in one day. Ten of these were inspections 
conducted by ministry inspectors and 26 were the 
work of external consultants. 

Insufficient time spent on inspections increases 
the risk that serious deficiencies will be missed, 
especially in older structures and bridges that have 
a history of problems. 


Changes in the Condition of Bridges 
between Inspections 


In general, if a bridge does not undergo any 
rehabilitation between inspections, its value on the 
Bridge Condition Index would decrease as it con- 
tinues to age and deteriorate. The rate of deteriora- 


tion is slower at first but accelerates as the bridge 
ages. In trying to predict future bridge rehabilita- 
tion needs, the Bridge Priority Tool automatically 
reduces a bridge’s Priority Index by 1.5% to 2% per 
year, depending on the age of the bridge. 

A significant increase or decrease in a structure’s 
Bridge Condition Index value between inspections 
raises questions: What are the possible reasons 
for the change, and what follow-up action should 
the Ministry take? When we compared the Bridge 
Condition Index numbers from current and previ- 
ous inspection cycles, we noted that there was an 
increase or improvement of five or more points— 
not a decrease as would be expected—for over 300 
structures. Recent rehabilitation work could only 
explain a few of these increases. Differences in the 
application of judgment on the part of the inspector 
was the reason most often cited for the increases 
that remained. | 

Conversely, we noted that for about 180 bridges, 
the Bridge Condition Index did not change at all 
between inspections. Our follow-up work indicated 
that the previous inspection reports had been car- 
ried forward for many of these bridges. Although in 
many of these cases there were photographs on file 
to indicate that an inspection had occurred, given 
that bridges do deteriorate over time, we believe 
that the absence of any explanation for why the 
bridge condition did not change warrants further 
follow-up. 

In 2006, one region re-inspected 41 bridges that 
previously had been inspected by external consult- 
ants. It found that almost 20% of the bridges had 
Bridge Condition Index values that varied between 
5 and 35 points from the initial inspection. Among | 
other things, the review noted that inspectors were 
not consistently applying inspection guidelines or 
verifying bridge inventory data. The region con- 
cluded that better in-house expertise was needed to 
monitor the work of external consultants. 


Bridge Condition Survey 


If an inspector feels that more detailed information 
on a structure is required than can be learned in 
the course of a visual inspection, he or she may 
request further investigation. For example, the 
presence of severe material defects or performance 
deficiencies in the individual elements of a bridge 
may necessitate further investigation, which, for 
concrete bridge components, is usually in the form 
of a bridge condition survey. In a condition survey, 
procedures that are more precise than visual 
inspection techniques are used to assess the extent 
of the defects and deterioration in a structure. For 
instance, the Ministry has conducted condition 
surveys on only about 5% of the province’s bridges 
in the last four years. Our expert indicated that, 
considering almost a quarter of the province’s 
bridges are in fair or poor condition, one would 
have expected the Ministry to use the more compre- 
hensive bridge condition survey more frequently. 


Agreements with Engineering Firms 


The regions that have outsourced the inspection of 
bridges have entered into individual agreements 
with the external firms they have engaged to do the 
work. We noted a number of substantial variations 
in the terms of these agreements. For example, one 
region had stricter requirements regarding experi- 
ence (the inspector had to have at least five years 
of bridge-design experience in addition to being a 
professional engineer) and scheduling (there was 
a maximum number of hours that could be spent 
inspecting bridges in one day, inspections could 
only be carried out during daylight hours, etc.). 


RECOMMENDATION 4 


To ensure that inspections are conducted in 
accordance with legislation, the Ministry of 
Transportation should establish a risk-based 
approach for the ongoing monitoring of inspec- 
tions. This approach should include: 


Bridge Inspection and Maintenance Tea 


@ assessing the reasonableness of the number 
of bridges that external contractors and min- 
istry staff report as having been inspected in 
any one day to ensure that thorough inspec- 
tions are being done; 

@ following up on any unusual changes in a 
bridge’s condition since the previous inspec- 
tion; and 

e identifying high-risk bridges that should be 
subject to more in-depth condition surveys. 
The Ministry of Transportation should also 

consider standardizing its agreements with 

engineering firms. At a minimum, these agree- 
ments should contain provisions regarding the 


experience and qualifications of staff assigned 
by the firm to conduct the inspections. 





Thorough and accurate inspections are neces- 
sary to ensure bridge safety. To enhance the 
assessment of bridge conditions, we have 
implemented the Auditor General’s suggestions 
to require ministry engineers to more clearly 
identify mandatory minimum timeframes for 
inspection and identify when unusual changes 
have occurred to the condition of a bridge. This 
will enhance the Ministry’s ability to take the 
appropriate follow-up action. 

Standardized inspection contracts have also 
been implemented that require specific experi- 
ence and qualification requirements for all 
contractors performing inspections. 


BRIDGE MAINTENANCE 


During the course of a bridge inspection, the 
inspector is to identify the bridge’s rehabilitation, 
repair, and maintenance needs. Future capital con- 
struction projects are to be included in the five-year 
capital work plan, whereas minor capital works in 
the $100,000 to $500,000 range are considered 
maintenance and are to be captured on a separate 
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list. Repairs to spalled concrete in columns or soffits 
and fatigue cracks in steel girders are considered 
maintenance work and would not appear on the 
Priority List and capital work plan. 

There are two types of maintenance work. 
Structural maintenance is work that requires engin- 
eering drawings. It is generally done to improve the 
structural capacity of a specific bridge element or 
on bridges in need of emergency repairs. Routine 
maintenance is usually preventative maintenance 
and minor repair work carried out by bridge crews 
aimed at prolonging the life of the bridge structure. 
All maintenance needs are to be recorded on an 
inspection form and forwarded to the mainten- 
ance crews of the responsible region for action, 
with urgent items flagged for immediate attention. 
When maintenance work is completed, the region 
is to confirm that the required work, especially 
all safety-related maintenance, was performed 
satisfactorily. 

We noted that regions did not always complete 
recommended maintenance work in a timely man- 
ner. In two of the three regions that we visited, 
only about one-third of the maintenance work 
recommended in biennial inspections was actually 
completed. The third region did not track the work 
being done, so we were not able to determine the 
number of maintenance recommendations that it 
had followed. 

We noted that the procedures for acting on 
maintenance recommendations resulting from 
biennial inspections varied considerably between 
the three regions. For instance, only one region 
made any attempt to prioritize its maintenance 
recommendations into categories such as “low 
priority,” “high priority,” and “immediate atten- 
tion.” This region was acting for the most part on 
recommendations that fell into the last category 
as well as a small percentage of its high-priority 
recommendations. 

Finally, although the Ministry had started tak- 
ing more of an asset-management approach to its 
maintenance activities, it had not yet developed a 
formal asset-management plan. Such a plan would 


set out the optimal time frame in which to carry out 
preventative maintenance as well as the most cost- 
effective approach for managing bridge assets over 
their life cycles. 


RECOMMENDATION 5 





The Ministry of Transportation should: 

e develop a formal asset-management plan as 
a basis on which to prioritize the preventa- 
tive maintenance of bridges; and 

® promptly carry out preventative mainten- 
ance, including the maintenance recom- 
mended in bridge inspections. 





All critical bridge safety needs are addressed 


immediately once identified by inspectors. In 
response to the Auditor General’s suggestion, 
we are currently creating a system to effectively 
prioritize maintenance work. This will be sup- 
ported by consistent documentation of inspec- 
tion results across the province. 

The Ministry is implementing a program to 
more efficiently allocate capital resources for 
roads and bridges. Comprehensive multi-year 
regional work plans for all provincial roads and 
bridges will clearly identify necessary rehabilita- 
tion, replacement, and preventative mainten- 
ance. These work plans evaluate a broad range 
of criteria, including bridge condition, as well as 
effective value for money. 


ONTARIO BRIDGE MANAGEMENT 
SYSTEM 


As indicated earlier, the Ministry uses the Ontario 
Bridge Management System (System) to keep 
information on all provincial bridges. In addition 
to calculating a Bridge Condition Index value for 
each bridge, the System generates information on 
inspections, such as when they are due, the date 
of their completion, the name of the responsible 


inspector, and the work that was recommended. 
This information is supported by photographs of 
defects observed during inspections. 

The System was developed about 10 years ago 

to enable the Ministry and its regions to follow a 
more disciplined approach to managing the bridge 
inspection process. To ensure consistency and avoid 
duplication of effort, the Ministry’s regions are sup- 
posed to use only the Ontario Bridge Management 
System. However, we found that each of the three 
regions we visited had been maintaining at least one 
additional local database. We noted the following 
issues: 

e The Bridge Management System did not have 
information on the rehabilitation history for 
almost one-third of the bridges that were 40 
years old or older. The Ministry confirmed 
that rehabilitation work had been done on 
some of these bridges in the last 12 years and 
detailed information was available in paper 
files or on the local database, yet none of 
this work had been entered into the System. 
Because the Bridge Priority Tool projects the 
next time rehabilitation work is due based on 
the information that has been entered into the 
System, the dates projected were inaccurate 
for work on some bridges. 

@ The System’s design caused it to perform 
some operations slowly. In some cases, when 
a user logged on to launch a detailed view, the 
System had to retrieve data for all sites in the 
region or province, which meant it took longer 
to perform the operation. The System was 
also slow in generating detailed inspection 
reports because it involved assembling data 
from a number of different tables. Regional 
staff tended to use local databases because 

‘they contained data on fewer sites and were 
therefore much faster. 

@ The System has limited reporting capabilities. 
For example, there are a number of standard 
summary reports that generate information 
based on the latest data, but it was not possible 
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to generate these reports for a specific year— 
even though the required data was stored in 
the database. For instance, users could not 
query the system to flag large fluctuations 

in Bridge Condition Index values between 
inspections. The summary reports were also 
restricted in format. Users could not vary the 
layout of the reports to suit their needs. 

The Ministry has made a number of upgrades 
to the System over the years. Despite these efforts, 
some of the above issues cannot be resolved 
because of the System’s age and design limitations. 


- RECOMMENDATION 6 


To make the Ontario Bridge Management Sys- 
tem more useful, the Ministry of Transportation 
should: 

@ ensure that the information on bridge. 
rehabilitation contained in the System is up 
to date; and 

@ assess whether the System meets users needs 
and whether there are cost-effective ways of 
improving its performance and capabilities, 
especially with respect to reporting informa- 
tion needed for rehabilitation and inspection 
purposes. 





The large amount of data included in the Bridge 


Management System must be comprehensive for 
it to be effectively used in decision-making. The 
Ministry is acting on the Auditor General’s sug- 
gestion to develop a business case for making 
further significant bridge-management system 
enhancements, which would strengthen the 
Ministry’s oversight of bridges in this province. 
We expect the business case to be completed 

in fall 2010. The new system, if implemented, 
would address all of the Auditor General’s 
findings. 
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PROCUREMENT AND CONTRACT 
MANAGEMENT 


Contract Selection Process 


The Ministry outsources almost all the work for its 
major road and bridge capital projects. The services 
involved in this type of work fall into three categor- 
ies: engineering design services, construction, 

and construction contract oversight. The Ministry 
generally followed a competitive selection process, 
but we noted some areas for improvement in its 
procurement of contracts for design services and 
construction contract oversight. 

The Ministry has developed two separate sets of 
evaluation criteria to rank the proposals it receives 
for design services and construction contract 
oversight. In the first type of evaluation, 50% of 
the bidding consultant’s score is to be based on 
its previous performance, and the remaining 50% 
is to be contingent on price. In the second type of 
evaluation, 50% of the score is to be based on the 
bidding consultant’s previous performance, 40% is 
to be based on the technical merits of its proposal, 
and 10% is to be contingent on price. We were 
informed that for straightforward, clearly defined 
projects, the first set of evaluation criteria would 
be used. For more complicated projects, where the 
consultant’s qualifications, approach, and ability to 
deliver are considered crucial elements, the second 
set of criteria would apply. 

We agreed that, for certain projects, qualita- 
tive considerations are as, if not more, important 
as price. However, for the second set of criteria 
in particular, the Ministry could not adequately 
support the weighting it had chosen to use—price 
was Virtually irrelevant given that it received only 
10% of the weighting. In these types of projects, 

a contractor pre-qualification process would have 
allowed the Ministry to learn more about contract- 
ors in advance and put more weight on pricing in its 
formal requests for proposals. 

The Ministry also was not clear about when and 
where each set of criteria would apply. The three 


regions we visited varied significantly in their appli- 
cation of the two sets of evaluation criteria. One 
region had decided to use only the first set of cri- 
teria, while the other two regions were using both. 

For nearly 60% of the contracts we reviewed, 
there were no more than two bidders. When there 
were no bidders, the Ministry ended up assigning 
the work to a consultant already engaged by it on 
another project. According to the Ministry, there 
has been a significant decline in the number of con- 
sultants bidding on design services and construc- 
tion oversight contracts because of consolidation 
in the consulting industry. Recognizing this, the 
Ministry prepared a business case in September 
2008 asking to increase its complement of staff so 
that it might reduce its dependence on external 
consultants. If approved, the Ministry would, 
although continuing to use consultants, gradually 
increase the number of projects done in-house over 
a five-year period. 


Price Estimates and Change Orders 


We noted that in over 60% of the contracts we 
reviewed for design services and construction con- 
tract oversight, the Ministry’s cost estimate differed 
significantly from that of the selected bidder. In 
many cases, the winning bid was 50% higher than 
the Ministry’s own initial estimate. 

We also noted significant change orders after 
contracts had been awarded. Change orders occur 
when the consultant performs work that was not 
included in the original agreement. This could be 
due to an unforeseen requirement for extra work 
or additional materials, or a change to the scope of 
work because of new information uncovered dur- 
ing the project. We noted addenda in about 75% 
of the contracts that we reviewed. In half of these, 
the added costs amounted to more than 50% of the 
original contract price. 


RECOMMENDATION 7 


To ensure value for money on major capital 
projects and fairness in its procurement process, 
the Ministry of Transportation should: 

@ review the application of its two different 
sets of evaluation criteria for requests for 
proposals to ensure that they are consistently 
applied across the regions; 

@ reassess the evaluation criteria in which 
the bid price is a relatively minor factor in 
selecting the winning bidder; and 

@ given the frequent significant variances 
between the Ministry’s estimated cost of a 
project and the bidder’s cost, examine its 
internal estimation process as well as the 
possible impact of the increased trend of 
relatively few bidders. 


The Ministry uses a fair, open, transparent, and 





competitive process to hire contractors, one that 
ensures good value for money. Our criteria for 
bid selection were developed by the Ministry in 
consultation with both industry and the Min- 
istry of Government Services. Price continues to 
be a determining factor in the selection of most 
successful bids. However, the Ministry’s experi- 
ence has been that for more complex projects, 
an emphasis on price during the design phase 
reduces the implementation of innovative and 
more efficient designs and drives construction 
costs higher. 

To ensure greater consistency and transpar- 
ency in how the Ministry selects contractors, 
detailed guidelines now clearly identify 
contractors’ responsibilities and project require- 
ments. A program is being implemented to 
closely monitor and evaluate the difference 
between estimated and actual design costs. 
Once it is complete, we will evaluate and make 
adjustments, where necessary, to our internal 
estimating process. 
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MUNICIPAL BRIDGES 
Condition of Municipal Bridges 


Each municipality is responsible for the bridges 

in its own jurisdiction. As well, there is no central 
database on the number of municipal bridges and 
their overall condition. There is also no legislation 
that provides any provincial ministry with the 
authority to oversee municipalities’ compliance 
with the legislated requirement that they conduct 
biennial inspections that assess the condition and 
safety of municipal bridges. 

One objective of our survey of municipalities 
was to gain an understanding of the municipal 
bridge inventory and how municipalities report 
on the required biennial inspections. The 73 
municipalities that responded to our survey were 
responsible for approximately 7,300 bridges. These 
bridges were, on average, older than the provincial 
bridges. They were 12 to 100 years old, with an 
average age of about 43 years. 

Almost all of the respondents said they engaged 
outside engineering firms to conduct bridge inspec- 
tions and used a variety of systems to keep track of 
municipal bridge and inspection data. For example, 
some have adopted the Ontario Bridge Manage- 
ment System and others rely mainly on data main- 
tained by external engineering firms, spreadsheets, 
and paper-based systems. 

Sixty-five (90%) of respondents indicated that, 
overall, their bridges were in good to fair condition. 
However, the definitions and systems used by muni- 
cipalities to classify bridges vary widely, so it was 
not possible to provide an accurate picture of the 
overall condition of municipal bridges in Ontario, 
nor was it possible to make comparisons between 
municipal and provincial bridges. 

Nevertheless, our survey indicated that muni- 
cipalities are finding themselves in a situation 
similar to that of the province as the need for 
significant rehabilitation becomes more pressing 
for many municipal bridges. The majority (85%) 
of respondents have a backlog of rehabilitation 
work. The urgency of the backlog varies, with 45% 


nN 
S 
oe) 
= 
S 
= 
7) 
® 
” 
= 
ra 
= 
e 
o 
te 
o 
at 
= 
& 
= 
oO 





22 4 2009 Annual Report of the Office of the Auditor General of Ontario 


a 
eS 
ae) 
= 
S 
= 
° 
o 
~” 
= 
rs 
=> 
e 
bot] 
— 
o 
ad 
2. 
c 
oS 
Oo 





of municipalities having a backlog of one to five 
years, 25% between six to ten years, and 10% over 
10 years. 

The primary sources of municipal revenue are 
residential and commercial property taxes, develop- 
ment charges, and user fees. Large and growing 
communities generally do not have such significant 
backlogs because their revenue bases are larger and 
their infrastructure newer. On the other hand, some 
municipalities have a large number of bridges but 
a relatively small population and revenue base to 
support the rehabilitation of bridge infrastructure. 
Figure 2 illustrates the extent of this disparity using 
a few examples from the results of our survey. 


Funding to Municipalities 
In recent years, the provincial and federal govern- 
ments have provided municipalities with funds to 
help them maintain their roads and bridges. Muni- 
cipalities can use the funds for infrastructure or 
spend them on other capital priorities. The decision 
is at the discretion of the municipality, depending 
on the terms of the grant program under which it 
received the funds. 

In 2004, the federal and provincial governments 
pledged a combined $596 million ($298 million 
each) over five years to improve Ontario’s muni- 
cipal infrastructure, of which $112 million was 
earmarked for bridges. In addition, in 2005/06 and 
2007/08, the Ministry of Transportation provided 
one-time grants of $400 million to municipalities 
for roads and bridges. Other one-time grants for 
improving municipal infrastructure were also made 


available through various provincial ministries in 
2007/08: $450 million under the infrastructure 
program and $1.1 billion under the Investing in 
Ontario Act. The decisions to make these grants 
available were often made close to the end of the 
fiscal year, with little advance warning. 

Many of the municipalities noted that, although 
they welcomed such one-time grants, this type 
of funding makes long-term capital planning dif- 
ficult. Because it takes time to properly plan capital 
projects, obtain council and environmental assess- 
ments, and follow the proper processes for approv- 
als and procurement, many municipalities were 
not able spend the money until long after they had 
received the grant. 

During our audit in spring 2009, we looked at a 
sample of municipalities that had received grants 
in June 2008. We found that half of them had yet to 
spend a large portion of the grant money they had 
received almost one year later. Some municipalities 
told us that the significant one-time grants they had 
received had actually increased project costs—the 
influx of requests for proposals from various local 
governments flooded the market with several 
projects at the same time, and with only a limited 
number of contractors available to do the work, bid 
prices tended to escalate. 

In addition, the province had little knowledge 
about the condition of the bridges in each muni- 
cipality and their maintenance and rehabilitation 
histories. The province allocated funds using 
demographic information, such as the size of the 
population and the network of roads. As such, the 





Figure 2: Municipal Capacity to Maintain Bridge Infrastructure 


Prepared by the Office of the Auditor General of Ontario 


Overall 






# of Condition of 











Backlog Backlog 





Population* Bridges (Smillion) (years) 
Municipality A 108,177 823 fair to poor 117.5 19.5 
Municipality B 62,563 242 fair 9.5 2 Sy, 
Municipality C 668,549 108 good nil n/a 
Municipality D 892,712 139 good nil n/a 


* As of 2006, according to Statistics Canada 


province may not be allocating funds to the areas of 
greatest need. 

On the other hand, some municipalities 
expressed concern that providing funds for only 
the bridges in poor condition, although urgently 
needed, penalizes municipalities that have consist- 
ently followed good asset-management practices 
in maintaining their bridges. Municipalities told 
us that requirements for better asset-management 
practices supported by more sustainable provincial 
funding are needed to ensure safety and maximize 
the lifespan of municipal bridges. 


Accountability 

We surveyed the accountability relationship in 
the management of municipal bridges in other 
Canadian provinces as well as six states in the 
United States. We found that Ontario and four 
other provinces have delegated the responsibility 
for the inspection and rehabilitation of local bridges 
to municipalities without making any provision 
for provincial oversight. The other jurisdictions we 
looked at either share or delegate the maintenance 
responsibility but still maintain an oversight role. 
For example, each state in the U.S. is expected to 
maintain an oversight role over the safety of its 
bridges through the National Bridge Inspection 
Standard. 

In 1993, the Government of Quebec transferred 
responsibility for the municipal road network 
to municipalities. The Commission of Inquiry 
that investigated the cause of the September 
2006 collapse of the de la Concorde overpass in 
Quebec found that there was ambiguity between 
the province’s Ministry of Transportation and its 
municipalities over who was responsible for the 
maintenance, repair, and replacement of bridges on 
the municipal road network. One of the Commis- 
sion’s recommendations was that Quebec’s Ministry 
of Transportation regain ownership of all bridges 
from municipalities with a population of less than 
100,000, or at least assume responsibility for their 
inspection, maintenance, and rehabilitation. In 
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January 2008, Quebec adopted the Commission’s 
recommendation. 

In Ontario, representatives from the province, 
the Association of Municipalities of Ontario, and 
the City of Toronto began working together in 
December 2006 to reach a consensus on a new fis- 
cal and service-delivery partnership between the 
province and the municipalities. This initiative, 
called the Provincial—Municipal Fiscal and Service 
Delivery Review, covered fiscal relationships, infra- 
structure, and the delivery of human resources. 
With respect to infrastructure, the partners agreed 
to launch a joint provincial—municipal process in 
fall 2008 to develop options for identifying respon- 
sibilities and funding arrangements for roads and 
bridges using recognized asset-management princi- 
ples. At the time of our audit, a working group with 
representatives from the province and the muni- 
cipalities was being established to follow up on 
the Review’s recommendations. Its objectives will 
include identifying municipalities with insufficient 
resources to maintain adequate levels of investment 
in roads and bridges, and developing a mechanism 
to provide them with extra assistance. Our survey 
results seem to indicate that this initiative is sup- 
ported by many municipalities. Accordingly, we 
have made some specific recommendations for both 
the province and the working group to consider. 


RECOMMENDATION 8 


To help ensure the safety and proper upkeep 

of municipal bridges, and as part of its current 

provincial—-municipal review, the Ministry of 

Transportation should work with municipalities 

and other stakeholders to: 

@ review practices in other large provinces and 
U.S. states with respect to oversight of muni- 
cipal responsibilities for bridge maintenance, 
with the aim of determining whether chan- 
ges to the current accountability relationship 
are required; 

e ensure that the condition of municipal 
bridges is consistently assessed, updated 
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every two years as required, and publicly 
reported; 

@ review the Ministry’s funding arrangement 
with municipalities to ensure that the funds 
provided are effective in sustaining the 
proper maintenance and rehabilitation of 
bridges; and 


@ promote good asset-management practices. 





The province and municipalities are jointly 


examining options around responsibilities and 
funding arrangements for roads and bridges. 


Currently, municipalities are responsible for 
bridges under their jurisdiction and the associ- 
ated inspections in accordance with the Public 
Transportation and Highway Improvement Act. 
Should further changes be warranted as a result 
of the review, we would certainly consider mak- 
ing some recommendations. | 

In the interim, we continue to work with our 
municipal partners to develop best practices for 
roads and bridges, focusing on development of 
an inventory of assets, classification of roads, 
and a review of how to manage these critical 
pieces of infrastructure. 


Chapter 3 
Section 


AIK: 





Background 





The Ministry of Consumer Services (Ministry) over- 
sees business and industry practices in Ontario’s 
consumer marketplace for the protection of con- 
sumers and public safety. It does so by establishing 
a regulatory and legal environment that protects 
consumers, educating the public and businesses 
about business standards and other relevant issues, 
dealing with complaints received from the public, 
monitoring and inspecting businesses, and enfor- 
cing compliance with various consumer protection 
laws and regulations. The Ministry has responsibil- 
ity for 27 consumer protection and public safety 
statutes. The primary legislation for which the 
Ministry directly monitors compliance includes the 
Consumer Protection Act, 2002 (CPA), the Collection 
Agencies Act, the Cemeteries Act, and the Film Clas- 
sification Act, 2005. 

The CPA gives the Ministry powers to dissemin- 
ate information for the purpose of educating and 
advising consumers; to provide information to 
consumers about the use of alternative dispute 
resolution techniques; and to enforce the CPA and 
other consumer protection legislation. The CPA 
covers consumer and business rights and their 
respective obligations, as well as requirements for 
warranties, unfair business practices, and other 


Ministry of Consumer Services 


Consumer Protection 


areas. In addition, businesses that fail to respond 
and/or are found to be in violation of a consumer 
protection statute may be listed on the Consumer 
Beware Database, which is available to the public 
on the Ministry’s website. 

The responsibility for a number of consumer 
and public-safety statutes for specific marketplace 
sectors has been delegated to eight designated 
administrative authorities (delegated authorities) 
for some time, as shown in Figure 1. The delegated 
authorities are not-for-profit corporations and each 
has a board of directors. The majority of directors 
are industry representatives, although the Minister 
can appoint some members to each delegated 
authority’s board. The delegated authorities use 
their industry and technical expertise to carry out 
the day-to-day functions of ensuring public safety 
and/or consumer protection in their industries 
by regulating and monitoring business practices, 
by inspection and enforcement, by dealing with 
complaints, and, in some cases, by administering 
industry-specific warranty or compensation funds 
to qualifying consumers. The Ministry monitors the 
performance and activities of delegated authorities 
and retains control over certain major decisions. 

The Ministry’s Policy and Consumer Protection 
Division (Division) consists of three branches: the 
Policy Branch, the Consumer Protection Branch, 
and the Sector Liaison Branch (which oversees the 
delegated authorities). In the 2008/09 fiscal year, 
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Figure 1: The Ministry’s Delegated Authorities, as of March 31, 2009 


Source of data: Ministry of Consumer Services 






Delegated Authority 
(Year Established) 


Board of Funeral Services 
(BoFS) 
(1914) 





Electrical Safety Authority 
(ESA) 
(1999) 


Ontario Motor Vehicle 
Industry Council (OMVIC) 
(1997) 





Real Estate Council of 
Ontario (RECO) 

(1997) © 

Tarion Warranty Corporation 
(Tarion) 

(1976) 





Technical Standards and 
Safety Authority (TSSA) 
(1997) 


Travel Industry Council of 
Ontario (TICO) 

(1997) 

Vintners’ Quality Alliance of 
Ontario (VQA Ontario) 
(2000) 





the Division had approximately 110 staff and oper- 
ating expenditures of approximately $12.6 million. 


Audit Objective and Scope 


The objective of our audit was to assess whether 
the Ministry (and, as applicable, the Ministry in 
partnership with its delegated authorities) had 
adequate authority, systems, and procedures in 


place to: 


Primary Legislation Administered and Key Responsibilities 
Funeral Directors and Establishments Act 


regulates funeral services; licenses 2,500 funeral directors and 600 
businesses 


Electricity Act, 1998 


regulates the use of electricity and electrical equipment; enforces 
the Ontario Electrical Safety Code; licenses almost 14,000 
electrical contractors and electricians 


Motor Vehicle Dealers Act, 2002 


regulates about 8,600 motor vehicle dealers and 23,800 
salespersons 


Real Estate and Business Brokers Act, 2002 
regulates 56,000 real estate brokerages, brokers, and salespersons 


Ontario New Home Warranties Plan Act 


administers a mandatory new home warranty program; registers 
5,800 builders; enrolled over 1.45 million homes 


Technical Standards and Safety Act, 2000 


regulates over 250,000 facilities, contractors, and workers for the 
boilers and pressure vessels, amusement and elevating devices, 
hydrocarbon fuels, and upholstered and stuffed articles industries 


Travel Industry Act, 2002 
regulates about 3,000 travel retailers and wholesalers 


Vintners Quality Alliance Act, 1999 
regulates VQA standards for over 100 registered wineries 







Annual 
Expenditures 
($ million) 


lhe 


73.6 


6.6 


9.0 


68.2 


46.0 


3.6 


i2 


the protection of consumers; and 


accordance with its mandate. 


public safety and consumer protection in 


# of 





Staff 


10 


400 


66 


84 


250 


360 


17 


@ ensure compliance with relevant legislation 
and Ministry policies that are established for 


@ measure and report on its efforts to achieve 


We conducted our audit work primarily at the 


and examining a sample of case files. Because 


Ministry’s two administrative offices in Toronto. 
Our work included reviews and analyses of the Min- 
istry’s procedures and guidelines, interviews with 
staff, reviews of relevant reports and documents, 


current legislation does not permit our Office to 


audit the delegated authorities, our work in that 
regard consisted of reviewing processes in use and 
information received by the Ministry for its over- 
sight responsibilities over delegated authorities, 

In addition, we met with senior management of 
two delegated authorities and a number of current 
and former ministry-appointed board members 

to discuss their views on delegated authorities’ 
governance, ministry oversight and accountability 
arrangements, performance, and consumer protec- 
tion and public safety issues. 

We also considered the recommendations that 
we made in our last audit of this program in 2003 
(Ministry of Consumer and Business Services, 
Policy and Consumer Protection Division), our 
2005 follow-up on the status of those recommenda- 
tions, and the related recommendations made to 
the Ministry by the Standing Committee on Public 
Accounts after its hearing on this section of our 
2003 report. 

We researched consumer protection programs 
and legislation in other provinces. In this regard, 
we met with management representatives of the 
Office de la protection du consommateur Québec 
(hereafter referred to as Quebec’s consumer protec- 
tion agency) to discuss their consumer protection 
programs. We also met with public consumer 
advocates to ask their opinions on consumer protec- 
tion issues and the Ministry’s role and activities. In 
addition, in June 2009 we engaged an independ- 
ent research firm to conduct a survey to assess 
the Ontario public’s awareness of the Ministry’s 
consumer protection programs and of consumer 
protection rights. 

We also engaged on an advisory basis the ser- 
vices of an independent expert with public-sector 
senior management experience in consumer protec- 
tion programs, from another province. 

Over the past several years, the Ministry’s Audit 
Services Team conducted reviews of several aspects 
of the Ministry’s consumer protection operations: 
the performance measurement framework estab- 
lished for the Technical Standards and Safety 
Authority (TSSA) and the Electrical Safety Author- 
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ity (ESA); a Risk Management Policy and Compli- 
ance Protocol established for cemeteries; and 
compliance with the government-wide purchasing 
card and travel expenses policies and procedures. 
These reviews and reported results allowed us to 
reduce the extent of our work in these areas. 





The Ministry has made progress in addressing many 


of the recommendations we made in our 2003 
audit for achieving consumer protection and public 
safety, especially with respect to recent improve- 
ments in its oversight of delegated authorities. 
Several changes to legislation have been made that 
have strengthened consumer protection, such as 
larger penalties and longer maximum sentences for 
illegal activities and broader coverage of activities. 
The Ministry has also conducted several targeted 
initiatives to promote compliance with consumer 
protection legislation by certain industries. None- 
theless, we noted the following areas where addi- 
tional action is required to ensure that consumer 
protection legislation is being adequately adminis- 
tered and enforced: 

e@ The Ministry needs to better promote its man- 
date and services to consumers. The 33,800 
inquiries and 6,000 written complaints the 
Ministry received during the 2008/09 fiscal 
year represented about the average numbers 
over the last seven years but are decreases of 
12% and 15%, respectively, from their peak in 
2004/05. The Ministry has not done work to 
assess whether the significant drop in consum- 
ers’ contact with it was due to decreased pub- 
lic recognition of the Ministry’s role or better 
practices by Ontario businesses. By contrast, 
although Quebec’s population is only 60% 
of Ontario’s, its consumer protection agency 
received over 250,000 consumer inquiries 
and complaints annually. In addition, our own 
independent external survey indicated that 
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the Ministry would not be among Ontarians’ 
top choices for resolving a complaint. 

The Ministry had exercised good controls over 
its registration and licensing processes in most 
areas to ensure that applicants meet standards 
for integrity, honesty, and financial viability, 
and that they comply with legislation for their 
industry. However, for collection agencies, 
debt collectors, and bailiffs, more rigour is 
required. 

Problematic industries and repeat offenders 
need to be more effectively dealt with by the 
Ministry. For instance, no significant improve- 
ment was achieved for certain industries 
(such as collection agencies, home repairs, 
car repairs, home furnishings, health and 
fitness clubs, and credit reporting) that 
remained on the Ministry's Top 10 Complaints 
list from 2000 through 2008. The Ministry 
had assigned a relatively low priority to 
complaints about harassment by collection 
agencies, even though they topped the list for 
almost 10 years and were the subject of over 
10% of all inquiries and complaints received 
during 2008/09. 

The Ministry has had four or fewer inspectors 
during the last several years, or roughly 

one inspector for every 100,000 businesses. 
Because of the limited inspection staff 
resources, in 2008/09, proactive visits were 
not made to business types covered under 

the CPA that were in the Top 10 Complaints 
list. As a direct result of the 6,000 written 
complaints received, only 148 inspections 

and educational field visits were initiated by 
the Ministry. The Ministry’s lack of inspection 
powers under the CPA, which covers most 
businesses, hinder it from effectively identify- 
ing consumer protection violations. Similar 
programs in other Ontario ministries, dele- 
gated authorities, and provinces were noted 
that had both more inspection staff resources 
and powers to access businesses’ records. In 
addition, others had legislation allowing them 


to assess administrative monetary penalties 
as a cost-effective alternative for dealing with 
less serious violations. 

e The Ministry has made some progress since 
our last audit in enforcing compliance by 
cemetery owners with reporting require- 
ments under the Cemeteries Act. However, the 
Ministry had not identified and recorded in its 
information system, and therefore planned no 
follow-up action or investigation on, a number 
of financial discrepancies that we identified, 
including differences ranging from a few 
thousand dollars to over $2 million between 
the trust fund balance as reported by the cem- 
etery owner and the actual funds held by the 
trustee. 

With respect to the delegated authorities, we 
noted that the Ministry is making progress in 
improving its oversight role and accountability 
relationship with them. However, there are several 
areas that still need to be addressed: 

e In 2004, the Standing Committee on Public 
Accounts recommended that a comprehensive 
review of the delegated authority initiative 
was required to ensure that public safety and 
consumers are being adequately protected 
under delegated authorities, and that mech- 
anisms are in place to ensure that delegated 
authority outcomes are being accurately 
reported. The Ministry launched the review 
on an urgent basis only after a tragic propane 
explosion occurred in Toronto on August 10, 
2008, and over four years after the Commit- 
tee’s recommendation to do so. The review 
was completed in May 2009 and at the time 
of our audit the Ministry was considering the 
recommendations made. 

e A good control established over the delegated 
authorities is that the Minister can appoint up 
to half of their board members. However, min- 
isterial appointees range from 25% to 33% of 
the boards’ composition. Boards were domin- 
ated by the industries they regulate. The 
Ministry has not encouraged greater balance 


on boards between representation by govern- 
ment, consumers, the public, and industry. 

e There is no provision for the Ministry to have 
full access to delegated authorities’ informa- 
tion on such matters as quality assurance 
programs, strategic plans, executive salary 
and compensation packages, and board min- 
utes. Although accountability arrangements 
with most delegated authorities were detailed 
and formal, the Ministry’s accountability 
agreement with Tarion, which dated back to 
2003, was not. Legislative amendments have 
not been made to clearly define the Ministry’s 
authority over Tarion. In addition, the Min- 
istry does not believe that it has a mandate 
to oversee how cost-effectively the delegated 
authorities are operating. 

We noted that only one performance measure on 
customer satisfaction is reported publicly to cover 
all consumer protection programs delivered directly 
by the Ministry, and we questioned whether it was 
a reliable and meaningful measure. 


Detailed Audit Observations 





CONSUMER PROTECTION PROGRAMS 
DELIVERED DIRECTLY BY THE MINISTRY 


Public Awareness of the Ministry’s 
Mandate and Consumer Protection 
Legislation 


Since our 2003 audit, we noted that the Ministry 
has had difficulties in promoting its mandate and 
services to consumers. Figure 2 shows that the 
33,800 consumer inquiries and 6,000 written 
complaints received by the Ministry in the 2008/09 
fiscal year represented approximately their average 
numbers during the period 2002/03 to 2008/09. 
In addition, the numbers of inquiries and written 
complaints had both decreased from their peak 

in 2004/05—by 12% and 15%, respectively. The 
Ministry had done no work to assess whether the 
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decrease represented a lack of public awareness of 
its consumer protection mandate or whether busi- 
nesses and others were doing a better job in their 
consumer relations. 

The Ministry relies extensively on its website to 
disseminate consumer protection information to 
the public. We noted that the number of times the 
Ministry’s website was accessed tripled from about 
7,800 visits in 2007 to 28,000 visits in 2008. None- 
theless, these overall numbers seem relatively low. 

By contrast, although Quebec’s population is 
only 60% of Ontario’s, from 2005/06 through 
2007/08 Quebec’s consumer protection agency 
received about 251,000 consumer inquiries and 
complaints annually, or about six times more than 
Ontario, and its website was accessed over one mil- 
lion times annually, or about 35 times more than 
Ontario’s. The consumer protection mandates of the 
two provinces differ in some ways (for example, the 
Ministry handles home renovation issues, whereas 
Quebec’s consumer protection agency does not; and 
Quebec’s consumer protection agency handles travel 
industry complaints, whereas in Ontario that is the 
responsibility of a delegated authority). But in our 
view, the differences do not explain the significant 
volume differences between Quebec and Ontario. 





Figure 2: Consumer Inquiries and Written Complaints 
Received by the Ministry’s Consumer Protection 
Branch, 2002/03-2008/09 


Source of data: Ministry of Consumer Services 


—@ average 
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The Ministry informed us that its communica- 
tion strategies generally target specific industries 
or businesses, but do not include large campaigns 
to promote either the Ministry’s mandate or con- 
sumer protection legislation. The Ministry has not 
conducted a formal assessment of the effectiveness 
of its communication and outreach activities or why 
its services are not used more extensively. 

The results of our survey conducted by an 
independent research firm also indicated that the 
Ministry could better promote awareness of con- 
sumer rights and its consumer protection programs 
and services. Some of the survey’s key findings 
were: 

@ Overall, almost two-thirds of respondents 
reported having experienced a consumer- 
related problem in the past five years, with the 
most frequently cited issues being harassment, 
misleading sales information, and a lack of 
transparency in contractual relationships. 
Over 50% of those who indicated that they 
had experienced a consumer-related problem 
said that they did not contact anyone for help. 

@ When asked who they would be most likely to 
contact if they could not resolve a consumer 
complaint issue with a company, respondents 
cited the Better Business Bureau (BBB), the 
legal profession, and someone in the media 
before the Ontario government. (The BBB is a 
not-for-profit public-service organization with 
offices serving communities and marketplaces 
across North America that accredits busi- _ 
nesses and upholds high standards for fair and 
honest business behaviour and for dealing 
with consumer complaints.) 

@ Among those who did contact someone for 
help, 23% indicated that they contacted the 
BBB. The Ontario government and the legal 
profession were each contacted by 11%, 
followed closely by the Canadian govern- 
ment, someone in the media, or a municipal 
government. 

e Less than half of respondents had used the 
Internet to get information about consumer- 


related problems, and only 3% of these 
respondents had used government websites to 
obtain this type of information. 

As discussed later in this report, we also noted 
that the Ministry has not conducted a general pub- 
lic survey similar to ours. Instead, it asks those who 
call the Consumer Protection Branch to rate the 
quality of and their satisfaction with its services. 
We were informed by the Ministry that it gener- 
ally receives high ratings in each of these areas. In 
contrast, our independent survey identified that 
one-third of respondents who had contacted the 
Ministry indicated that it was not helpful. 


: RECOMMENDATION 1 





To ensure that there is adequate public aware- 
ness of the Ministry’s consumer protection 
mandate and complaint services, the Ministry 
should: 

e consult with other jurisdictions that have 
significantly more activity and recognition 
by the public to see if there are any best 
consumer-protection practices that can be 
applied in Ontario; 

@ assess its outreach and education programs 
with a view to identifying changes needed to 
make them more effective; and 

e establish mechanisms for regularly assessing 
the general public’s awareness of consumer 
rights and the Ministry’s programs. 





The Ministry agrees that cross-jurisdictional 
consultations are valuable and believes that 


both public awareness of the Consumer Protec- 
tion Branch program as a source of information 
and help along with public knowledge of basic 
consumer rights and obligations are key to pro- 
tecting Ontario consumers from falling victim to 
unfair and inappropriate business practices. 

The Ministry does and will continue to con- 
sult with other provinces to learn from their suc- 
cessful public outreach/educational programs 
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and explore mechanisms for assessing consumer 7 Siete Sap Can Ha Spree peoatet 


awareness. The Ministry will explore the feas- 
ibility of adopting and adapting these strategies 
to Ontario consumer needs. The Ministry notes 


citizenship, landed immigrant documents, or 
other documents showing eligibility to work 
in Ontario. 


: : We also noted that the Ministry continued to 
that some of its consumer education products, : 


such as the Smart Consumer Calendar, have 
been adopted by other provincial jurisdictions. 


renew collection agencies’ licences even when they 
had been the subject of numerous past complaints. 
The Collection Agencies Act gives the Ministry the 
powers to refuse to issue a new registration or to 
revoke or suspend an existing one if the Ministry 
Registration and Licensing believes that the business “cannot reasonably be 
Industry-specific legislation requires that certain Dae - We nena psp ouaie Es pe 
: 5 duct of business” or its past conduct indicates that it 
businesses be registered and licensed by the Min- 

istry. As of March 31, 2009, the Ministry registers, 


licenses, or appoints more than 12,000 individuals 


“will not carry on business in accordance with law 
and with integrity and honesty.” For example, we 
: found about 20 collecti ies that had each 
and businesses. During the 2008/09 fiscal year, the een es gee aa ; ‘ " oe 
averaged from 20 to more than 460 inquiries and ° 
complaints annually in 2002/03 through 2008/09. 


All of their licences were renewed and active 


Ministry collected approximately $5.6 million in 
registration and licensing fees. 


We noted that the Ministry generally had 

; ‘ over the last two years, even though there was no 

exercised good controls over its registration and ; 

documentation of any follow-up during the licens- 

licensing processes for the cemetery, theatre, and ; vis 

: b ‘ ing renewal process to assess the legitimacy of the 

video-store industries; however, improved controls j 

complaints. 

were needed over the processes used for collection Hide ; 

; on . The Ministry informed us that it has revoked 

agencies, debt collectors, bailiffs, and assistant : : : ] 

ft i 5 ; licences for serious and repeat violations, mak- 

bailiffs. A key requirement for the registration and ; as 
ing these decisions on a case-by-case basis. For 


example, between 2002/03 and 2008/09, it 
revoked the licences of five collection agencies; two 


licensing processes is to ensure that applicants 
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meet standards for integrity, honesty, and financial 


viability, and that they comply with the consumer es i : : 
ei } y P M ; ‘ debt collectors, 10 bailiffs, and nine assistant bail- 
protection and business obligation requirements of 


eth tra! iffs. We agree with the Ministry that the number of 
the legislation for their industry. We found that the 


complaints should not be the only basis for revok- 
files we reviewed had little or no documentation nee 
to ety ing or withholding a collection agency’s licence, 
to demonstrate that the business or individual met nae 
because complaints might not be justified or be all 


these requirements, including: ee 

r that serious in nature. 

e for collection agencies, debt collectors, and 
assistant bailiffs, whether the applicant’s 


credit history had been checked; 


However, especially when the volume of com- 
plaints is significant, some follow-up should be 


tre required. The Ministry had not established a policy 
e for collection agencies and bailiffs, the infor- ae bit 
or guidelines that could be used to help assess how 
mation on whether the requiredtrust account . ~.. ; 
inquiries and complaints are to be considered dur- 

was established; ; : : 
. ; ing the registration and licensing renewal process, 
e for collection agencies, whether financial ; 
to ensure that it treats all businesses consistently, 

statements were obtained; ; 3 ; 
at and to identify the level of non-compliance by a 
e whether criminal background checks on the ; 
aN k business that would trigger a licence review and 
principals involved were ever conducted; and 
possibly revocation. 
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RECOMMENDATION 2 


To ensure that its registration processes meet 
legislative requirements that only businesses 
that demonstrate financial responsibility 

and honesty and integrity are registered and 
licensed, the Ministry should: 

@ review the procedures, documentation 
requirements, and quality control processes 
that its staff must follow to conduct a proper 
and complete review of an application; and 

e establish a policy and guidelines for staff to 
use that would require due consideration of 
the number and types of complaints about 
an applicant. 





The Ministry agrees that periodic review of 


registration/licensing procedures, documenta- 
tion requirements, and quality control processes 
is advisable and will conduct a review of the 
licensing review process, starting with the col- 
lection agencies program, this fiscal year. 

The Ministry notes that improvements to 
the Consumer Affairs Tracking System (used 
by its staff to record consumer inquiries and 
complaints and any action taken by the Ministry 
to help educate the consumer or resolve the 
complaint) to support better documentation are 
already under way in the cemeteries area. 

The Ministry has completed a number of 
reviews and revisions of the operational poli- 
cies applying to regulatory decisions. This is an 
ongoing process, and the Ministry will work to 
develop and document policies that establish 
criteria to “flag” chronic violators as candidates 
for further administrative action. 


Dealing with Consumer Complaints 


The CPA gives the Ministry the mandate to receive 
consumer complaints (whether or not the complaint 


involves conduct that represents an offence under 
consumer protection legislation) and to make 
inquiries, gather information, and attempt to medi- 
ate or resolve complaints. The Ministry’s action 
depends on the nature and severity of the complaint 
and on whether there is an allegation of an offence. 
In most phone inquiries, consumers are advised 

of their rights and businesses’ obligations, and 
informed of steps they can take themselves, such 

as seeking legal assistance or going to Small Claims 
Court. Complaints relating to industry sectors that 
are regulated by one of the eight delegated author- 
ities are referred to the relevant delegated authority 
to handle. If consumers request ministry follow-up 
action, they typically file their complaint in writing 
to authorize the Ministry to take further action. 

The Ministry cannot force businesses to com- 
pensate consumers. For the approximately 6,000 
written complaints the Ministry receives annually, 
the Ministry may make a phone inquiry, conduct 
a mediation process between the consumer and a 
co-operating business, and/or initiate an inspection 
(if the business is required to be registered with the 
Ministry) or conduct an “educational field visit” 
(for other businesses). For serious allegations of an 
offence under consumer protection legislation for 
unfair business practices, such as failure to disclose 
key contract requirements and to refund deposits 
for work not completed, the Ministry initiates a 
formal investigation, which can lead to prosecution. 
In addition to fining or jailing offenders, courts 
can also order them to pay compensation or make 
restitution to victims. In the 2008/09 fiscal year, 
the Ministry’s actions resulted in about $437,000 in 
full or partial voluntary refunds from businesses to 
consumers and about $428,000 from enforcement 
efforts and prosecutions. 


Recent Initiatives 
Since our 2003 audit, several changes to legisla- 
tion have strengthened consumer protection. For 
example, the Consumer Protection Act, 2002 (which 
consolidated six previous consumer protection laws 


and came into force in 2005) provides consumers 
and businesses with new rights, responsibilities, and 
remedies for a fair, safe, and informed marketplace. 
The new CPA stipulates, for instance, increased 
penalties for illegal activities: the maximum 
sentences were increased to two years less a day; 
and maximum fines were doubled to $50,000 for 
individuals and more than doubled to $250,000 for 
corporations. In addition, it covers leasing agree- 
ments, purchases made over the Internet, and other 
agreements that were not covered in the old legisla- 
tion. Since 2005, new legislation or amendments to 
existing legislation has been introduced to address 
marketplace concerns or improve consumer protec- 
tion regarding gift cards issued by retailers, payday 
loans, title fraud for property, real estate sales, col- 
lection agencies, the travel industry, identity theft, 
electrical safety, technical standards and safety, new 
home purchases, and motor vehicle purchases. 

In addition, recently the Consumer Protection 
Branch has conducted several targeted initiatives 
to promote compliance with the consumer protec- 
tion legislation by certain industries. For example, 
the Ministry initiated action to review service 
contracts used by fitness clubs and wireless phone 
providers; discuss rules of conduct with the Can- 
adian Association of Movers; enter into a one-year 
joint pilot project with the Ministry of Revenue 
to educate car repair shops and video retailers to 
promote increased compliance with the legislation; 
and visit certain companies that sell gift cards and 
payday loans to educate them on new legislative 
requirements. 


Information Systems 
Since our 2003 audit, the Ministry has made signifi- 
cant improvements to its Consumer Affairs Tracking 
System (CATS), which is used by its staff to record 
consumer inquiries and complaints and any action 
taken by the Ministry to help educate the consumer 
or resolve the complaint. The Ministry can build on 
this initiative by considering the following areas 
where further improvements could be made: 


Consumer Protection 


e There are at least 20 categories established in 


CATS for complaint-handling staff to record 
the industry or type of services that the public 
inquired or complained about, which, for 
example, allow the Ministry to generate the 
annual Top 10 Complaints list that it reports 
to the public. However, for staff who conduct 
inspections and educational field visits, CATS 
limits their recording of the industry in which 
the activity took place to only four categor- 
ies: cemeteries; debt recovery; theatres; 

and, for all other businesses and industries, 
“CPA.” The “CPA” category was the most often 
used: in 2008/09, of the 342 inspections and 
educational field visits conducted, 145 (42%) 
fell into that category. This high use of the 
“CPA” category inhibits the proper analysis 

of violations and of education, inspection, 
and enforcement activity. For instance, the 
number of educational field visits conducted 
for areas with high complaints, such as fit- 
ness clubs and car repair shops, could not be 
determined from CATS, nor could the visits 
recorded under “CPA” be compared with the 
20 categories recorded for inquiries and com- 
plaints for further analysis. We were informed 
that, due to the new legislation regarding 

gift cards and payday loans, the Ministry has 
recently started to record in CATS its inspec- 
tions and educational field visits for these 
specific industries. 

In 2008/09, the Consumer Protection Branch 
closed just over 39,800 inquiries and com- 
plaints, of which about 23,700 (60%) were 
closed by Ministry staff after having provided 
information to the consumer. However, staff 
are not required to record in CATS what type 
of information they provided, the nature of 
the consumer-related concern raised, or the 
action recommended by the Ministry to be 
taken. In addition, no information is captured 
in the system about the types of violation or 
contraventions of the laws (such as improper 
or misleading signage or contracts) found by 
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inspections. This information was generally 
available on the individual inspection reports, 
but could not be summarized from CATS. 
Such information would be useful for assess- 
ing inquiries and complaints and for targeting 
areas where future educational activities 
should be directed. 

e@ The Ministry reported on its website that it 
handles, on average, about 55,000 complaints 
and questions each year. However, we noted 
that in 2008/09, although all of the about 
6,000 written complaints were recorded in 
CATS, only about 70% of the calls (about 
33,800 of 48,800) were recorded there. For 
the remaining 30% of calls, Ministry staff 
recorded no information (such as whether 
they were not consumer-protection related, 
or the caller called the wrong number or 
Ministry). 


RECOMMENDATION 3 


To enhance the ability of staff to use the 

information recorded in the Consumer Affairs 

Tracking System to analyze consumer issues by 

the type of industry and the type of inquiry or 

complaint, the Ministry should: 

® capture information on its inspections and 
educational field visits by industry and viola- 
tion type and on the type of information 
provided for the public inquiries; and 

@ ensure that the nature of all inquiries and 
calls is input into the system. 





The Ministry agrees that improvements to the 


inspection module of the Consumer Affairs 
Tracking System (CATS) database are necessary 
to more fully report and analyze compliance 
issues. The Ministry will make programming 
changes to better report on issues, viola- 

tions, and business types. The Ministry will 

also review the quality of information being 
recorded on CATS with respect to inspections 





of registered/ licensed businesses to ensure that 
CATS provides optimum information on compli- 
ance to management. 

We agree that recording the nature and 
subject of inquiries is key to determining topical 
compliance issues and marketplace trends. The 
Ministry will work to ensure that staff record 
appropriate details of all inquiries insofar as 
they are relevant to our consumer protection 
mandate. 


Problematic Industries and Repeat Offenders 
We found that more progress was needed to deal 
with problematic industries and to ensure that 
repeat offenders are more effectively dealt with by 
the Ministry. For instance, we reviewed the Top 10 
Complaints list published by the Consumer Protec- 
tion Branch and found that certain industries (such 
as collection agencies, home repairs, car repairs, 
home furnishings, health and fitness clubs, and 
credit reporting) had remained on the list from 
2000 through 2008, suggesting that no See 
improvement was achieved. 

For example, collection agencies had topped the 
list for almost 10 years. In 2008/09, about 4,200 
inquiries and complaints (over 10% of all inquiries 
and complaints) were related to collection agen- 
cies. Although the Consumer Protection Branch 
conducted an average of 40 inspections and five 
investigations in this industry annually over the 
last five years, collection agencies remained the top 
complaint. We were also informed that inspections 
did not address harassment issues (which repre- 
sented, on average, about 30% of the closed written 
complaints in the last three years). Given the lim- 
ited resources devoted to inspections, the Ministry 
had assigned a relatively low priority to harassment 
complaints in relation to other complaints (such 
as those involving cemetery trust funds and home 
renovations) because they involve low monetary 
values and low public-safety risk. 


The Ministry also informed us that it had 
difficulties in gathering evidence of harassment 
by collection agencies. Our research identified 
that Quebec has somewhat stronger legislation 
governing debt collectors, including a regulation 
requiring that all communications by the collector 
with the debtor, such as telephone calls and let- 
ters, must be recorded in a register. An investigator 
can subsequently use the register and any other 
information to more thoroughly investigate and, if 
warranted, prosecute the collection agent. No simi- 
lar requirement exists under Ontario’s Collection 
Agencies Act. 

We also noted that certain businesses had high 
numbers of complaints and were known to the Min- 
istry as repeat offenders, but the Ministry’s enforce- 
ment efforts had not been effective in bringing 
about a change in their business practices. When the 
Ministry’s efforts do not result in obtaining compli- 
ance and permanent changes to a business’s oper- 
ations, there is a risk that the business will come to 
believe that the ramifications of getting caught from 
time to time are an acceptable risk they take and 
a “cost of doing business.” We noted the following 
examples of where these situations may exist: 

@ The Ministry had received written complaints 
against one fitness club relating to automatic 
renewals, billing disputes, cancellation of 
contracts, and other allegations of violations 
under the CPA. The fitness club was contacted 
in May 2007 by a special Ministry enforce- 
ment project targeting fitness clubs to review 
the legality of their contracts. The Ministry’s 
efforts appear to have had some but not 
complete success in reducing the number of 
written complaints against this club from 65 
in 2007/08 to 33 in 2008/09. 

@ One vacation club had 131 written complaints 
filed from 2005/06 to 2007/08, or about 44 
complaints on average per year. The Ministry 
initiated an investigation in each of the years 
from 2005/06 through 2007/08: two of 
these investigations resulted in either refunds 
to consumers or prosecution, and one was 
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ongoing at the time of our audit. However, 
the Ministry still received a further 22 written 
complaints about this business in 2008/09. 

@ One lawn-care business had an average of 
33 written complaints each year between 
2004/05 and 2006/07. An investigation 
launched in 2004/05 resulted in obtaining 
refunds to consumers; another, launched 
in 2006/07, was ongoing at the time of 
our audit. The complaints were generally 
about the same unfair practices, including 
unsolicited services provided and negative- 
option marketing (that is, charging consumers 
for goods or services they did not request). 
However, in 2007/08 and 2008/09, the Min- 
istry received, respectively, 24 and 31 further 
similar written complaints, which suggests 
that the behaviour has persisted. 

We were informed that the Ministry had no for- 
mal plans to further address industries in Ontario’s 
Top 10 Complaints list, such as movers, home 
repairs/furnishings, health and fitness clubs, credit 
reporting, or time shares. We were advised that 
addressing issues in these industries would require 
further policy analysis and possibly legislative 
changes and additional staff resources. 


- RECOMMENDATION 4 





To ensure that it can effectively deal with indus- 

tries and businesses that incur high numbers 

of and/or repeated consumer complaints, the 

Ministry should: 

@ conduct research to identify best practices 
in other provinces that can be applied in 
Ontario to improve compliance by certain 
industries and businesses; and 

e identify industries and businesses that per- 
sistently incur a high number of consumer 
complaints, assess the effectiveness of its past 
enforcement activities used against these 
problematic industries and businesses, and 
establish effective education and enforce- 
ment strategies for dealing with them. 
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The Ministry agrees that cross-jurisdictional 
consultations are valuable; it consults with 
other provinces when developing reform 
proposals. The Consumer Protection Branch 
will initiate a discussion forum with other 
Canadian consumer protection agencies on 

best compliance-management and -enforce- 
ment practices. Ministry staff will conduct an 
environmental scan of the best practices and 
techniques of other provincial jurisdictions with 
a view to enhancing marketplace compliance by 
businesses operating in Ontario. 

The Ministry regularly identifies “problem 
sectors” and allocates its compliance inspection 
resources, using a risk-assessment framework 
for proactive inspections and a priority-setting 
protocol for reactive inspections. Ministry 
investigations are assigned priority using a 
points-based system that accounts for complaint 
volumes by issue/sector and by specific busi- 
nesses. The Ministry will assess and revise, as 
appropriate, its risk-based enforcement strategy 
for industries and businesses that have historic- 
ally been problematic for consumers and about 
which the Ministry has persistently received 
high numbers of complaints. 

The Ministry will explore the development 
of effectiveness indicators against which it 
can assess its compliance and enforcement 
performance. 


Inspections and Educational Field Visits 
When warranted by complaints the Ministry 
receives from consumers, the Ministry may carry 
out inspections or educational field visits to 
determine whether further ministry action, such 
as investigation and enforcement, is required. The 
Ministry can also use inspections or educational 
field visits proactively, to educate businesses and try 
to reduce non-compliance with legal requirements 


and deter unfair practices. Inspections can be made 
only at registered businesses—that is, those that are 
covered by separate legislation (such as cemeteries, 
collection agencies, or theatres and video stores). 
All other businesses receive only educational field 
visits, because they fall under the CPA, which 

does not grant inspection powers to the Ministry’s 
inspectors. In 2008/09, the Ministry conducted 197 
inspections and 145 educational field visits. 

In our 2003 audit, we noted that the Ministry 
did not deploy its inspection resources based either 
on any formal assessment of risk to the public and 
consumers or on the number of complaints received 
for each of the industries it monitors. In 2006, the 
Ministry introduced an annual risk assessment pro- 
cess to better allocate its inspection staff to areas of 
higher consumer risk. Factors considered as part of 
the risk assessment include public safety, changed 
rules under new legislation, the risk of significant 
consumer financial loss, and the number and type 
of complaints. 

Notwithstanding this new risk assessment pro- 
cess, our review identified that too few staff resour- 
ces are made available for inspections and that the 
Ministry’s limited powers under the CPA, which 
covers most businesses, reduces its effectiveness in 
identifying consumer protection violations. 

Regarding the Ministry’s staff resources, we 
noted the following: 

@ Between 2002/03 and 2007/08, the Ministry 

had four inspectors for the entire province 
to cover both registered and non-registered 
businesses (roughly one inspector for every 
100,000 businesses). For half of 2008/09, 
vacancies reduced the number of inspectors 
to two. The Ministry has been aware of its 
staff shortages for some time. As stated in its 
2005/06 Results-based Plan, “the Ministry 
notes that its complement of inspectors is 
dramatically lower than in the past due to 
constraints. At one time, the Branch included 
seven regional cemeteries inspectors and 
five regional theatres inspectors, plus a vary- 
ing number of ‘generalists’. The number of 


inspectors is now four, to cover everything 
that the Branch regulates. This is inadequate 
to prevent further erosion of compliance 
standards.” 

As indicated in Figure 3, the number 
of inspections and educational field visits 
dropped significantly, from 1,723 to 342, 
between 2002/03 and 2008/09. Most of the 
large drop occurred in 2003/04, when the 
Ministry dramatically reduced its inspections 
of video retailers following the concerns we 
raised in our 2003 audit about over-inspecting 
these types of businesses at the expense of 
under-inspecting others. However, there 
was an overall downward trend in the total 
numbers of inspections and educational field 
visits being carried out over the period. We 
were informed that the decline was partly due 
to the higher number of cemetery inspections . 
(which can take more than twice as long as 
other inspections) being carried out as a result 
of the new risk assessment process. 
In 2008/09, the Ministry conducted 227 pro- 
active visits: 60 of these visits were to educate 
businesses that sell gift cards and make payday 
loans on new legislative requirements; and the 
remaining 167 proactive visits were used to 
conduct inspections and target registered busi- 
nesses such as cemeteries, collection agencies, 
and theatres/video stores. Because of limited 
staff resources, no proactive visits were made 
to other non-registered businesses covered 
under the CPA (including those in the Top 10 
Complaints list, such as car repair shops and 
home furnishings and appliances retailers). 
The new risk assessment process is used 
to allocate existing resources, but it does 
not determine the number of inspections 
or educational field visits that should be 
conducted for each industry sector to obtain 
a reasonable sample and representation of 
the industry’s level of compliance. Nor had 
the Ministry conducted any other formal 
assessment to determine the number, nature, 


Consumer Protection rant 3 


eee EER ON RRO ee eee Oe 
Figure 3: Number of Inspections and Educational Field 


Visits by the Ministry's Consumer Protection Branch, 
2002/03-2008/09 


Source of data: Ministry of Consumer Services 
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Note: The Ministry uses only four categories to segregate its inspections and 
educational field visits by industry type, as follows: 

debt recovery includes inspections of collection agencies, debt collectors, 
bailiffs, and assistant bailiffs; 

cemeteries includes inspections of cemeteries, crematoria, and 
mausoleums; 

CPA/other includes educational field visits made to all businesses not 
covered by industry-specific legislation; and 

theatres includes inspections of movie theatres and film and video 
distributors and retailers. 


and extent of inspection resources needed 
to adequately address the consumer risks 
in various industries. Rather, inspections 
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were initiated and assigned based largely on 
available inspection resources, with those 
resources being allocated based on Ministry 
priorities at the time. We noted that only 148 
inspections and educational field visits were 
initiated in 2008/09 as a direct result of the 
approximately 6,000 written complaints the 
Ministry received. However, no log was main- 
tained to track the number of requests made 
by the Ministry’s complaints-handling staff 
for inspections or educational field visits and 
whether the inspection staff actually carried 
out those requests. This information would be 
useful for determining if internal demand for 
inspections and educational field visits was 
being met. 
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e The Ministry’s delegated authorities with 
inspection activities had greater numbers 
of inspectors than the Ministry, yet were 
responsible for fewer businesses. For example, 
OMVIC had nine inspectors for the motor 
vehicle industry and TICO had five inspectors 
for the travel industry. Other provinces with 
smaller populations and fewer businesses had 
greater inspection resources for their con- 
sumer protection programs: British Columbia 
had five inspectors, and Quebec’s consumer 
protection agency has trained the 35 con- 
sumer protection officers in its 11 regional 
offices to perform basic inspections, and they 
currently devote 40% of their time to these 
inspections. Similarly, Ontario’s Ministry of 
Labour currently has 146 permanent and 23 
temporary employment standards officers to 
investigate complaints and conduct proactive 
inspections regarding businesses’ employment 
practices. 

As noted earlier, only at registered businesses 
can the Ministry’s inspectors conduct actual inspec- 
tions. Under the separate laws governing such 
businesses, the Ministry’s inspectors can enter 
a licensee’s premise at any reasonable time; ask 
the business to produce documents or items that 
may be relevant to the inspection; and inspect and 
remove documents or items relevant to the inspec- 
tion. All other businesses are covered by the CPA, 
which provides the Ministry’s inspectors only with 
the right to “make inquiries, gather information 
and attempt to mediate or resolve complaints.” 

If a serious violation is alleged or suspected, the 
Ministry may conduct a formal investigation, which 
requires that a search warrant be obtained based 
on probable grounds that a violation has occurred 
in order for the investigator to obtain the informa- 
tion he or she would need to properly investigate. 
Otherwise, the Ministry’s inspectors can only make 
“educational field visits” to CPA businesses: they 
have no authority to request and inspect books and 
records, even if numerous complaints have been 
made against that business or that type of business 


(as is the case, for example, with fitness clubs, vaca- 
tion clubs, and lawn-care companies). 

By comparison, we noted that increased pow- 
ers of inspection are included under the legisla- 
tion administered by the Ministry’s delegated 
authorities for regulated businesses and under the 
Ministry of Labour’s Employment Standards Act for 
any business. In both British Columbia and Quebec, 
consumer protection legislation provides inspection 
powers, including the power to enter a business’s 
premises “at any reasonable time” and to “inspect, 
audit, or examine any record, goods, or other 
thing or the provision of services in the premises.” 
Manitoba’s Consumer Protection Act gives consumer 
services officers the right to access premises during 
normal business hours “where there are reason- 
able and probable grounds to believe that those 
premises contain specific documents, correspond- 
ence and records relevant to the complaints” and to 
“make copies, or take extracts from, the documents, 
correspondence and records.” 


RECOMMENDATION 5 





To expand its coverage and capabilities for its 
inspection activities for the protection of con- 
sumers, the Ministry should: 

@ conduct a formal assessment of the number 
of inspection staff resources it should have 
to adequately fulfill its mandate and ensure 
comprehensive coverage; and 

e explore the need to obtain increased legisla- 
tive authority and powers for its inspectors, 
consistent with those in other consumer 
protection organizations in Ontario and 
other provinces, that would allow them to 
more efficiently and effectively deal with 
consumer complaints and identify potential 
consumer protection violations. 





The Ministry will use a risk-based approach to 


assess the compliance resources necessary to 
fulfill its mandate. The Ministry has increased 


its inspection complement and recruited two 
inspectors to bring its inspection team to full 
strength of five inspectors. In addition, in an 
effort to improve its regulatory reach, the Min- 
istry is working with the Ministry of Revenue 
to test the use of Revenue staff to increase the 
number of educational field visits focused on 
motor vehicle repair shops and video retailers. 
This approach has helped to increase our pres- 
ence in these areas of the marketplace. 

The Ministry agrees that increased inspec- 
tion powers would assist in furthering consumer 
protection within the province of Ontario. The 
Ministry will explore the feasibility of changing 
legislation to achieve this. 


Investigations and Enforcement 
As Figure 4 indicates, between 2002/03 and 
2008/09, there have been some large declines in 
the Ministry’s investigation activities and outcomes. 
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Staff resources remained about the same, but we 
were advised that the investigation process had 
become more complex and lengthy, particularly 
since there is a trend of laying a greater number of 
charges for each case, and that new requirements 
had increased the workload involved in processing 
court restitution orders. However, we are not aware 
of any formal assessment having been conducted 
by the Ministry on the effectiveness of its enforce- 
ment activities and use of available enforcement 
measures. 

Current enforcement measures available to the 
Ministry include revoking a licence for a registered 
business; issuing a Compliance Order for a business 
covered under the CPA; posting a business’s name 
on the Consumer Beware Database; and initiating 
prosecution and seeking court orders. Since our 
2003 audit, the Ministry had revoked between four 
and 28 licences each year, but it had issued only 
three Compliance Orders over the entire period. 
The Ministry also issued deficiency letters to 
individuals and businesses, but these were mostly 


a 


Figure 4: Investigation Activities and Enforcement Outcomes by the Ministry’s Consumer Protection Branch, 


2002/03-2008/09 


Source of data: Ministry of Consumer Services 
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Change 
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# of investigations closed 179 132 136 15 113 107 114 Bae fab) 
7 ENG ald 147 114 144 162 122 163 158 7 
businesses charged 

# of convictions 125 93 238 265 159 365 161 29 
length of jail time and 

probation ordered by 733 344 578 836 724 706 474 | (35) 
courts (months) ger: 
settlements negotiated . 
by investigators priorto 595,706 481,436 77343 111,288 133,508 103,957 100,283 (83) 
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* This excludes refunds ($437,645 in 2008/09) obtained each year through the Ministry's complaints-handling process and mediation services prior to any 


investigations. 
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for mediation purposes and not binding to the 
individuals or companies. We were informed that 
revoking a licence may not be an effective approach 
for certain common issues: for example, it may not 
help ensure a cemetery owner’s compliance with 
the annual reporting requirement. Both issuing a 
compliance order and prosecution require an oner- 
ous amount of the Ministry’s time and resources. 

In a number of other North American jurisdic- 
tions, as well as in several other Ontario ministries, 
administrative monetary penalties have been intro- 
duced as an alternative to prosecution for certain 
offences. Under this arrangement, violators who 
have not committed a criminal offence are assessed 
financial penalties. If a case is appealed, the 
administrative process followed is quicker and less 
costly than going through the courts. In addition, 
certain other ministries can issue small fines under 
Part I of the Provincial Offences Act; commonly 
referred to as tickets, these can be a cost-effective 
and more immediate means of dealing with less 
serious violations. The Ministry has neither of these 
more cost-effective enforcement options available 
in enforcing consumer protection laws and would 
require changes to its legislation before they could 
be introduced. 

In contrast, legislation governing the Ministry’s 
delegated authorities permits them to impose 
administrative monetary penalties. Other minis- 
tries’ programs with powers to impose administra- 
tive monetary penalties include those involving 
employment standards, environmental protection, 
food safety, and forest management. 

In addition, we noted that the Ministry had not 
formally assessed the effectiveness of its Consumer 
Beware Database in promoting compliance since it 
was introduced in 2005. In this regard, we noted 
the following: 

e@ Our independent survey found that fewer 
than one in 10 respondents had heard of the 
Consumer Beware Database. Of these, only 
about 10% had used it. However, when a 
description of the Ministry’s website and of 
the Consumer Beware Database was provided 


to respondents, 96% said it is very likely or 
likely that they will use the information. 

e Although almost 10,000 Internet users 
accessed the database in 2008, by compari- 
son, almost 60,000 Internet users annually 
access Quebec’s equivalent database. 


~ RECOMMENDATION 6 


To help ensure that the Ministry’s enforcement 
efforts are both timely and cost-effective in 
achieving compliance and in deterring future 
violations of consumer protection laws, the 
Ministry should: 
e consider introducing more expeditious 
and effective enforcement tools, including 
administrative monetary penalties and tick- 
ets, for violations that either do not warrant 
criminal prosecution or are less serious; and 
e undertake periodic reviews, including 
researching best practices in other similar 
organizations, of its investigative program, 
enforcement measures, and the Consumer 
Beware Database, to assess their effective- 
ness and identify areas for improvement. 





The Ministry will consider current enforcement 
tools and evaluate whether additional tools 


are required for the treatment of less serious 
violations. 

Inspection enforcement tools have improved 
under the new Payday Loans Act, 2008, which 
allows inspectors to impose Administrative 
Monetary Penalties (AMPs) in response to a 
range of infractions under the Act. The Ministry 
will carefully evaluate the efficacy of AMPs to 
determine their potential for enhancing compli- 
ance in other sectors; implementation of AMPs 
in other sectors will require legislative reform. 

The Consumer Protection Branch will 
initiate a discussion forum with other Can- 
adian consumer protection agencies on best 
compliance-management and -enforcement 


practices. We will conduct an environmental 
scan of best practices and techniques of other 
provincial jurisdictions with a view to enhancing 
the Consumer Beware Database function and 
marketplace compliance by businesses operat- 
ing in Ontario. 

The Ministry agrees that interagency 
consultations are valuable; enforcement staff 
continue to participate extensively in several 
interagency forums. The Ministry will review 
the best practices of investigative programs in 
other jurisdictions. 


Cemeteries’ Trust Accounts 


The Cemeteries Act sets out the obligations of cem- 
etery owners and the rights of consumers when 
buying cemetery graves, cremation services, and 
interment services. Ontario has over 5,000 regis- 
tered cemeteries, which are operated by approxi- 
mately 2,400 owners; about two-thirds of the 
cemeteries have active sales. The Ministry monitors 
cemetery owners for compliance with certain finan- 
cial requirements, such as a requirement to deposit 
between 15% and 40% of the sales of interment 
spaces into trust fund accounts. The income from 
these accounts is intended to support the long-term 
cost of maintaining the cemeteries (headstones, 
grounds, and buildings). 

Given that consumers have paid hundreds of 
millions of dollars that have been deposited into 
trust funds, cemetery owners are required to file 
annual returns within three months after the 
cemetery’s fiscal year-end and are to include trust 
account statements. The return must be certified 
complete and accurate by both the owner and an 
independent trustee. Cemetery owners are also 
required to file, within six months after the cem- 
etery’s fiscal year-end, audited financial statements 
for care and maintenance trust funds containing 
more than $500,000 and for pre-need trust funds 
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(those relating to pre-arranged burial services) of 
more than $100,000. 

Ministry staff are responsible for reviewing, on 
a timely basis, the annual returns from cemetery 
owners to detect errors, omissions, and instances of 
non-compliance with filing requirements. The mon- 
itoring of such returns, especially in relation to trust 
fund accounts, is intended to minimize the risk that 
consumers’ payments are not deposited into the 
required trust accounts or that insufficient amounts 
are deposited. The other significant risk involves a 
cemetery having insufficient resources for care and 
maintenance purposes. In these cases, the cemetery 
may have to be turned over to the municipality to 
be maintained at the expense of local taxpayers. 
About 150 cemetery sites have been transferred to 
municipalities since 1999. 

In our 2003 audit, we found that less than half 
of cemetery owners had filed their required annual 
returns to the Ministry, and for those that did, the 
Ministry had not processed or adequately reviewed 
the returns to ensure proper accounting for the 
trust funds. For our current audit, we noted that 
the Ministry has made some progress in improving 
its monitoring of cemeteries’ financial reporting 
requirements and in ensuring that cemetery owners 
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comply with legal requirements. 





For example, in 2006 the Ministry established a 
“Risk Assessment Guide for the Monitoring, Com- 
pliance and Enforcement of Cemeteries in Ontario” 
for trust funds, and used that guide to prioritize 
inspection and investigation activities. From 2003 
to 2008, the Ministry initiated 35 investigations of 
cemetery owners, which had resulted in four pros- 
ecutions by the time of our audit. 

However, we concluded that the Ministry needs 
to take more effective action to deal with cemetery 
owners that do not fully comply with their annual 
filing requirements and to obtain compliance. For 
example, for the reporting years 2003 through 
2007, we noted that from 7% to 13% of all owners 
did not submit the required annual report each 
year. For 2007, more than 260 cemetery owners 
who operate one or more active cemetery sites had 
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not filed a return as of February 2009, which was 
more than a year overdue. Almost 90% of these 
delinquent owners were in the Ministry’s “small” 
classification: that is, they had fewer than 10 bur- 
ials per year and less than $100,000 in their trust 
fund accounts. However, we did note that two cem- 
etery owners had not filed an annual return since 
1992, including one “small” and one “large” (which 
would have had more than 10 burials per year and 
up to $500,000 in its trust fund). Ministry staff 
had tried to contact both owners numerous times 
since 2005 and had initiated inspections in 2008; 
however, as of the end of our audit, it still had not 
obtained compliance from them, nor had it issued 
compliance orders to the owners. 

For cemetery owners that do file returns, Min- 
istry financial review procedures need to improve 
to ensure that returns are properly assessed and 
that action is taken to address financial discrepan- 
cies. Our review of the Ministry’s Consumer Affairs 
Tracking System (CATS), which is used to track 
information from the returns, noted that for about 
160, or 8%, of the over 2,000 cemetery owners 
that operate the almost 3,000 active cemeteries, 
there was key information missing that would be 
necessary for verifying that owners’ reported fund 
balances matched funds held by trustees for any 
given year. 

In addition, for returns filed for the years 2003 
through 2007, CATS indicated that Ministry staff 
had identified “deficiencies” in almost 1,200 
instances when reviewing annual returns. Ministry- 
identified deficiencies could include balance 
discrepancies, errors, or missing reports or informa- 
tion. However, we found that not all of the deficien- 
cies were being recorded in the system. Our sample 
included 30 instances where discrepancies existed 
between the owner’s reported funds and the trust- 
ee’s bank account, but only seven of these cases had 
been identified by Ministry staff in CATS as having 
deficiencies. The amounts of the unidentified dis- 
crepancies where the owner’s balance was greater 
than the trustee’s balance ranged from a few thou- 
sand dollars to $2.4 million. Unless a deficiency is 


identified in CATS, the Ministry takes no follow-up 
action with the cemetery owner to investigate these 
differences. As of February 2009, we also noted 
that about one-quarter of the 1,200 deficiencies 
identified in CATS remained unresolved. 


~ RECOMMENDATION 7 


To ensure that cemetery owners comply with 

legislative reporting requirements and that 

funds are accounted for and sufficient for the 

proper long-term care and maintenance of cem- 

eteries, the Ministry should ensure that: 

e all annual returns are filed by all cemetery 
owners; and 

e timely and effective action is taken to 
enforce reporting requirements, to properly 
assess reports received, and to follow up on 
and resolve financial discrepancies identified 
on returns. 

In view of the significant demand that cem- 
etery legislation places on the Ministry’s limited 
staff resources, the Ministry should also explore 
the option of having cemetery legislation 


administered by a delegated authority. 





The prevalence of volunteer operators for small- 
scale cemeteries in this sector makes risk assess- 
ment complex and enforcement challenging. 
The Ministry will work to ensure that all annual 
returns are filed by cemetery owners. 

The Ministry recognizes that timely and 
effective action is important within the cem- 
etery sector and agrees to ensuring improved 
time frames for the resolution of financial dis- 
crepancies identified on returns. 

There is a provision within the Safety and 
Consumer Statutes Administration Act, 1996 
that allows for the cemeteries regulation to be 
administered by a delegated authority. 


MINISTRY OVERSIGHT OF DELEGATED 
AUTHORITIES 


After the 1996 passage of the Safety and Consumer 
Statutes Administration Act (SCSAA), Ontario estab- 
lished five designated administrative authorities 
(delegated authorities) to permit certain industries 
to undertake regulatory functions that had previ- 
ously been administered directly by the Ministry. 
At the time, the Ministry already had in place two 
similar arrangements for new home warranties and 
funeral service providers. In 2000, VOQA Ontario 
was established under its own legislation (see 
Figure 1). Key reasons that the delegated author- 
ities were established by the government were the 
assumption that they can be more responsive to the 
needs of the marketplace and of maintaining high 
marketplace standards because of their interest in 
the industries they regulate, and to reduce the size 
and cost of government. 

The SCSAA prescribes the key accountability 
relationship requirements for the five delegated 
authorities it established. For example, it requires 
the establishment of an administrative agreement 
between the Ministry and each delegated author- 
ity; requires the tabling in the Legislature of each 
delegated authority’s annual report; allows the 
Minister to appoint members of each delegated 
authority’s board of directors, as long as the minis- 
terial appointees do not constitute a majority of the 
board; and grants powers to the Minister to termin- 
ate the delegation. In contrast, the legislation estab- 
lishing Tarion does not specify its accountability 
requirements to the Ministry. 

Ministries are also required to follow the 
Accountability Directive established by Manage- 
ment Board Secretariat in 1997 for relationships 
with external service providers, including not-for- 
profit organizations such as delegated authorities. 
Three key elements are required for effective 
accountability: i) defining expectations, roles and 
responsibilities, and managing consistent action; 
ii) reporting on and monitoring performance; and 
lii) reviewing performance against expectations 
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and taking corrective action when required. This 
includes requirements for controls and verification 
procedures for both operations and finances and 
the right to conduct independent audits. 

Figure 5 summarizes the key expectations in the 
relationship between the Ministry and delegated 
authorities. 


Accountability and Oversight Arrangements 


Appropriate oversight is needed to ensure that 
services to the public are delivered cost-effectively, 
including when the services and key responsibilities 
for ensuring consumer protection and public safety 
are delegated to other organizations on the govern- 
ment’s behalf. The Ministry’s oversight relationship 
with each delegated authority is predicated on find- 
ing the right balance between appropriate high-level 
oversight on the one hand, and not micromanaging 
on the other. Delegated authorities must be allowed 
the autonomy to run their day-to-day operations 
without the constant involvement of ministry 
managers; but ministry management must ensure 
that an effective accountability relationship is in 
place and that sufficient, useful, timely, and credible 
information is being received and assessed to ensure 
that the public is getting the appropriate level of 
service in a cost-effective and timely manner. 

In our 2003 audit, based on our review of the 
information gathered by the Ministry and of the 
Ministry’s monitoring activities, we concluded that 
the Ministry did not have adequate assurance that 
public safety and consumers were being properly 
protected by delegated authorities. Our audit indi- 
cated that the Ministry did not have proper mechan- 
isms in place to ensure that outcomes reported by 
the delegated authorities were reliable. We also 
found that the Ministry’s monitoring efforts were 
inadequate. 

As a result of our 2003 audit and the Ministry’s 
continuing efforts to improve its accountability 
framework over delegated authorities, the Ministry 
made the following changes to its systems and 
procedures: 
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@ established an annual report tracking system 
to ensure that annual reports are received on 
time and Ministry feedback is provided; 





Figure 5: Key Expectations of the Ministry and Delegated Authorities 
Prepared by the Office of the Auditor General of Ontario 


Responsibility 


sector-specific legislation 
and regulations, industry 
standards and code of 
conduct 


governance 


organizational planning and 
accountability 


Role of Ministry 


establishes and amends legislation and 
regulations to adjust or change marketplace 
rules, and to establish delegated authorities’ 
mandate, powers, and responsibilities 


Minister may appoint up to 50% of 

board members, which may include 
representatives of consumers, industry, or 
government 


establishes administrative agreement 

with board that clarifies responsibilities, 
accountability arrangements, and 
expectations for conduct and performance 
reporting 

oversees delegated authority’s performance 
to protect the public interest and promote 
public safety and consumer protection 


reviews multi-year business plan and 
annual audited financial statements 


reviews annual report and tables it in the 
Legislature 


requests additional reports or information 
quarterly or as necessary from delegated 
authority 

Minister, Deputy Minister, and ministry staff 
meet with board and senior management 
several times each year to discuss key 
issues 


measures on a quarterly basis, and for mon- 
itoring performance trends and gaps against 


Role of Delegated Authority 


advises Minister by recommending 
legislative and regulatory proposals, or on 
matters of an urgent or critical nature 
develops industry standards and code of 
conduct 


board appoints majority of board members 
and establishes governance processes, 
including elections, annual meetings, voting 
procedures, bylaws, etc. 

board establishes corporate organizational 
structures, staffing needs, control 
frameworks, and administrative and 
operational policies 

responsible for overseeing all day-to-day 
administrative and regulatory activities 


develops multi-year business plans 
Outlining objectives, planned initiatives, and 
performance measures 


board publishes an annual report to the 
public and Ministry on its achievement of 
its mandate and key accomplishments 


obtains an annual financial audit 


fees approves fee-setting process for delegated establishes fee structure in accordance with 
authority fee-setting process and collects and retains 
fee revenues to fund operations 
compliance and enforcement no ministry role e registers and licenses businesses and 
individuals 


@ provides education and training 


® carries out inspections, investigations, 
and prosecutions, and issues sanctions 
for violations (e.g., revoking or suspending 
registrations or licences, issuing fines) 

@ reviews and mediates consumer complaints 
and may maintain a member-funded 
general compensation fund for reimbursing 
consumers 





the performance commitments made by the 
delegated authorities; and 

e established a Minister’s appointees tracking 

system that outlines the number and percent- 
age of Minister appointments on each Board, 
and their attendance at board meetings. 

The Ministry has also worked to establish and 
update its accountability agreements with dele- 
gated authorities. The Minister signed its first letter 
of accountability with Tarion in 2003, with several 
subsequent letters covering further agreed-upon 
changes. In 2005, the administrative agreements 
with all delegated authorities that fall under the 
SCSAA were updated, with the revised agreements 
specifying, for example, reporting requirements for 
business plans and annual reports. 

For our current audit, we noted that the Ministry 
has taken several good initiatives and has made 
progress in improving its accountability relation- 
ship with the delegated authorities; however, the 
Ministry needs to further strengthen its oversight 
and the delegated authorities’ reporting require- 
ments to protect consumers and the public interest. 
The following sections detail our observations and 
concerns. 


Formalizing Accountability Relationships 
with Tarion 


The Ontario New Home Warranties Plan Act (which 
was last amended in 2006) still does not include 
specific requirements for an accountability frame- 
work between the Minister and Tarion similar to 
that established by the SCSAA for other delegated 
authorities. Therefore, the Ministry has followed 

a separate path for establishing its accountability 
relationship with Tarion, and this has been a slow 
and negotiated process. The 2003 letter of account- 
ability signed by Tarion and the Minister was the 
first time Tarion formally acknowledged that it is 
fully accountable to the Minister. However, the 
letter of accountability was much less detailed than 
the administrative agreements established for other 
delegated authorities. The letter did not include or 
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specify the roles and responsibilities of both par- 
ties; any requirement regarding the submission or 
contents of a business plan; the details that should 
be included in an annual report; the fee-setting 
process and criteria; and the payments by Tarion to 
the Ministry for its oversight, which other delegated 
authorities pay. 

Thus, since 2003, Tarion’s accountability 
arrangements have lacked many of the require- 
ments that other delegated authorities had to fulfill. 
More recently, since 2006, following several letters 
from the Minister to Tarion—including concerns 
raised by the Minister about the high number of 
consumer complaints against Tarion—we noted 
that significant improvements have been made 
to Tarion’s reporting and accountability relation- 
ships with the Ministry. These improvements 
have included the completion of new homeowner 
surveys; more ministerial appointees on Tarion’s 
board; the sharing of Tarion’s business plan and 
strategic plan with the Ministry; the provision 
of additional quarterly performance informa- 
tion on service, claims, complaints, staffing, and 
governance issues; and changes to enhance public 
transparency (such as holding its first public annual 
general meetings in April 2009, and increased 
disclosure in Tarion’s annual report). At the time of 
our audit, Tarion had complied with almost all of 
the requests made by the Minister in 2008, and we 
were informed that the remaining requested chan- 
ges were in progress. 


Review of Accountability Relationships 


Following its review of our 2003 audit, the Stand- 
ing Committee on Public Accounts recommended in 
2004 that a comprehensive review of the delegated 
authority initiative was required to ensure that 
public safety and consumers are being adequately 
protected under delegated authorities, and that 
mechanisms are in place to ensure that delegated 
authority outcomes are being accurately reported. 
The Ministry’s response to the Committee at the 
time was that it had retained a consulting firm to 
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complete an evaluation of the delegated authorities 
in 2001, and it had worked with the delegated 
authorities to implement the recommendations 
arising from that evaluation. These recommenda- 
tions included improving communication with the 
public; evaluating stakeholder feedback; enhancing 
public education and awareness; establishing 
outcome-based performance measures; improving 
customer services; and better risk management. 
The Ministry stated that it would continue to work 
with the delegated authorities to make legislative, 
regulatory, and government improvements, based 
on ongoing reviews. 

The Ministry advised us that no further com- 
prehensive reviews were undertaken as a result of 
the Standing Committee’s 2004 recommendation. 
Instead, the Ministry had focused on improving 
its accountability arrangements with each of its 
delegated authorities. In addition, as a result of 
concerns we raised in 2003 regarding the reliability 
of the TSSA’s reported outcomes, the Ministry 
worked with the TSSA to improve controls over its 
data integrity. 

On August 10, 2008, a tragic propane explo- 
sion occurred in Toronto. At the completion of our 
audit, the formal investigation of the incident and 
the propane retailer was ongoing. In addition, the 
Ministry launched an independent review of pro- 
pane safety in Ontario. The registration, licensing, 
and inspection of propane retailers fall under the 
TSSA’s authority. At the time of the incident, public 
concerns were also raised about the quality of infor- 
mation reported by the TSSA on registered propane 
retailers, and this resulted in a further internal 
ministry review of TSSA’s board governance. 

In response to the incident, on August 28, 

2008, the Ministry established an expert panel to 
conduct a comprehensive review of the legislative 
and regulatory framework for the safe storage, 
handling, location, and transport of propane. On 
November 13, 2008, the Minister announced plans 
to implement all 40 recommendations made by the 
expert panel. On May 28, 2009, the government 
also introduced legislation designed to improve the 


TSSA’s accountability and transparency and further 
strengthen the province’s technical safety system. 
The proposed legislation would require the TSSA 
to appoint an independent Chief Risk and Safety 
Officer to report annually and publicly on how the 
TSSA is meeting its public safety mandate; provide 
the Minister with the authority to guide the TSSA’s 
strategic focus by issuing policy directives and to 
appoint the Chair and Vice Chair from the members 
of the TSSA’s board of directors; and provide our 
Office with access to the TSSA’s records should the 
Auditor General choose to conduct an audit. 

In addition, in October 2008 the Ministry 
launched the sort of review of its delegated 
authorities that the Standing Committee had 
recommended it undertake four years previously. 
The study was designed to be undertaken in two 
phases, to allow for the two public-safety delegated 
authorities—TSSA and ESA—to be given a higher 
priority in the first phase, with the second phase 
covering the six consumer protection delegated 
authorities. The scope of the review included three 
main elements: Governance and Accountability, 
Stakeholder/Public Relations, and Performance. 

The Ministry informed us that it was unable 
to follow traditional competitive procurement 
requirements for hiring a consultant to conduct this 
review, which, according to the Management Board 
of Cabinet Procurement Directive established in 
November 2007, would have required that at least 
five pre-authorized consultants from the govern- 
ment’s Vendor of Record (VOR) listing be invited 
to submit proposals. In this regard, the govern- 
ment’s VOR listing for the “Program Evaluation 
and Performance Measurement Services” category 
lists 30 consultants. We noted that the Ministry, 
instead, followed a “restricted-competitive” pro- 
curement process—an approach permitted by the 
procurement Directive when circumstances war- 
rant it, the contract is less than $500,000, and the 
Deputy Minister approves it. (Note: for amounts 
of $500,000 up to $1 million, approval from the 
Supply Chain Leadership Council, which is a senior 
management executive committee established by 


Management Board of Cabinet, is required). The 
Ministry supported its decision in this regard based 
on the urgency of the review (as part of the Min- 
istry’s response to the propane explosion); the need 
for a high level of confidentiality due to intense 
media scrutiny and public criticism of the delegated 
authority model; and the need for specialized 
consultant expertise in the area of public-sector 
governance and accountability. We noted that 

four consultants were invited to submit proposals, 
and only two responded with submissions. Only 
one of the four invited consultants was from the 
pre-authorized vendor-of-record list. In November 
2008, after an internal process to review the bids, 
the Ministry selected to perform the review a firm 
that was not on the VOR listing that bid a maximum 
cost of $499,335, even though the other consultant, 
which was on the VOR list, had bid about $150,000 
less. 

Phase one of the review was completed in March 
2009, and phase two was completed in May 2009. 
Overall, the review was supportive of the existing 
delegated authority arrangements, but it made 
a number of suggestions for strengthening the 
efficacy of the current model in areas such as cor- 
porate and regulatory governance, the Ministry’s 
accountability tools and oversight, stakeholder 
engagement, co-operation among the delegated 
authorities, and performance measures. In addi- 
tion, the consultant recommended improvements in 
areas such as inspection and enforcement activities, 
information systems, data quality control and 
assurance processes and procedures, and informa- 
tion disclosures. 
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Ministerial Appointees to Delegated 
Authorities’ Boards 


A good control established over the delegated 
authorities is that the Minister can appoint mem- 
bers to their boards, which may include repre- 
sentatives of consumers, industry, or government. 
Minister-appointed board members protect the 
Ministry’s interests by their direct involvement in 
corporate decisions and activities and can provide 
a consumer perspective to the boards, which are 
all dominated by the industry they regulate. A 
good practice implemented by the Ministry since 
2006 is to provide annual training to each Minister- 
appointed member. We noted the following areas 
for maximizing the benefits the Ministry receives 
from Minister-appointed board members and for 
balancing the representation of interests on boards: 
@ The SCSAA enables the Minister to appoint up 
to 50% of the members of delegated author- 
ities’ boards of directors. In our 2003 audit, 
we noted that board members who were 
independent of the industries being regulated 
were significantly under-represented: minis- 
terial appointees represented on average only 
16% of the boards’ total composition. Since 
that time, as part of the Ministry’s efforts to 
improve consumer and ministry representa- 
tion on the Tarion and TSSA boards, several 
additional ministerial appointments have 
been made. As noted in Figure 6, however, 
ministerial appointees constitute only from 
25% to 33% of the boards’ compositions. 
e Delegated authorities are not-for-profit cor- 
porations and have Ministry and statutory 
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Figure 6: Composition of Board of Directors for Selected Delegated Authorities, as of April 1, 2009 


Source of data: Ministry of Consumer Services 


ESA OMVIC RECO _ Tarion TICO = TSSA* Total 


total board members 12 
# of ministerial appointees 3 
% of ministerial appointees on the board 20 


12 12 15 15 14 foe 80 
3 Ss # 4 4 22 
25 25 33 Ze 29 27 
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* By September 2009, the TSSA intended to increase the number of Minister-appointed board members to six, and to decrease the total number of board 


members to 13, bringing the proportion of ministerial appointees to 46%. 
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mandates to protect public safety and/or 
consumers. Each of the boards has established 
its own process and criteria for selecting and 
appointing board members other than those 
appointed by the Minister. However, in the 
absence of any legislative requirement or 
direction from the Ministry, each of the dele- 
gated authorities has decided to have industry 
representatives form the majority of its board 
members, and the Ministry has not encour- 
aged greater balance between representation 
by government, consumers, the public, and 
the industry. The current industry domin- 
ance could lead to either a real or perceived 
industry bias for decisions made by delegated 
authorities. For example, Tarion, which has a 
stated mandate to protect Ontario’s new home 
buyers, has established a requirement that the 
majority of its board members (eight out of 
15) be appointed by the Ontario Home Build- 
ers’ Association (OHBA), which represents 

its industry’s interest. Furthermore, Tarion 
permits OHBA to have an observer (typically 
OHBA’s president) at all board meetings, 
which is an unusual business practice. 

There were three senior ministry staff, includ- 
ing two with direct responsibility for oversee- 
ing delegated authorities, appointed to board 
positions. As board members, ministry staff 
participate in the board’s decision-making 
processes and receive substantial information 
on the delegated authorities’ operations. In 
2004, the Supreme Court of Canada stated: 


“The statutory fiduciary duty requires 
directors and officers to act honestly 
and in good faith vis-a-vis the corpor- 
ation. They must respect the trust and 
confidence that has been imposed 

on them to manage the assets of the 
corporation in pursuit of the objects 
of the corporation. They must avoid 
conflicts of interest with the corpora- 
tion.... They must maintain the confi- 


dentiality of information they acquire 
by virtue of their position. Directors 
and officers must serve the corpora- 
tion selflessly, honestly, and loyally.” 


To help staff deal with any conflict-of- 
interest situation, we were informed, the 
annual training provided by the Ministry to 
Minister-appointed board members helps to 
address this possible situation and makes the 
Ministry’s interest paramount. Ministerial 
appointees, including ministry staff, are 
required to sign comprehensive confidential- 
ity agreements with delegated authorities. 
As a result, ministerial appointees are not 
empowered without permission of the board 
to discuss with the Ministry their discussions 
at the delegated authority’s board meetings 
or share any information they obtain while 
serving as a board member. This can create a 
difficult conflict situation for ministry board 
members. 


Access to Delegated Authorities’ 
Information 


The Freedom of Information and Protection of 
Privacy Act (FIPPA) applies to Ontario’s provincial 
ministries and most provincial agencies, boards, 
and commissions, as well as community colleges, 
universities, and Local Health Integration Net- 
works. FIPPA requires that the government protect 
the privacy of an individual’s personal information 
existing in government records; but it also gives 
individuals the right to request access to govern- 
ment information, including general records and 
records containing their personal information. 
Because the delegated authorities are not bound by 
FIPPA, however, the public has no right to access 
their information. 

We noted that the Ministry also has no right to 
access the delegated authorities’ records and, as 
stated in the previous section, Minister-appointed 
board members cannoi disclose information to the 


Ministry about delegated authorities’ activities due 
to the confidentiality agreements they sign. The 
administrative agreements state that “all records 
obtained from any source, created, or maintained 
by the [delegated authority] in the course of 
carrying out its delegated administration are the 
property of the [delegated authority].” We noted 
that the administrative agreements revised in 2005 
specified additional information (such as details 
on business objectives; performance measures on 
compliance, efficiency and effectiveness; and finan- 
cial details) that the delegated authorities should 
provide in their business plans and annual reports. 
However, there remain limitations on the Ministry’s 
access to information unless the delegated author- 
ity chooses to share its information. For example, 
the Ministry does not regularly request or receive 
information on: 

e board and advisory committee meeting 
minutes; 

@ quality assurance programs and the results of 
any reviews commissioned by the delegated 
authority to examine its programs and key 
areas, such as inspection activities and data 
quality; 

@ multi-year strategic plans that detail a dele- 
gated authority’s long-term key priorities and 
activities; 

@ reports made to the audit committees by the 
external financial auditors regarding any con- 
cerns over financial and operational internal 
controls and other financial matters; 

@ executive or staff salaries and other 
compensation; 

e employee and travel-related expense reim- 
bursement policies; 

@ use and cost of consultants; and 

e staffing resources in key areas, such as inspec- 
tion and enforcement, call-centre representa- 
tives, and senior management complements. 

The Ministry needs appropriate and timely 

information to ensure that the delegated author- 
ities’ boards of directors are adequately discharging 
their fiduciary responsibilities and establishing 
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quality assurance mechanisms for their systems and 
procedures to reduce key risks. A review of board 
minutes, for instance, would indicate significant 
matters that are brought to the boards’ attention 
and decisions made to address any concerns. 

For example, the TICO had been monitoring the 
precarious financial status of a large travel retailer 
for several months before the company suddenly 
terminated its operations on April 15, 2009, leav- 
ing its customers stranded abroad. We understand 
that the Ministry was unaware of the financial and 
consumer risk of this company’s continuing until its 
collapse was made public. 

Similarly, the Ministry could use information on 
salaries and other compensation packages to execu- 
tives and staff, on the use and cost of consultants, 
and on employee and travel-related expenses to 
ensure that the delegated authorities use their rev- 
enues in an appropriate manner for a quasi-public- 
sector organization. 

The Ministry does not provide any direction 
in its administrative agreements with delegated 
authorities to ensure the prudent use of their rev- 
enues and due regard for economy and efficiency 
with respect to their expenditures. Our discussion 
with Ministry staff overseeing delegated authorities 
indicated that imposing requirements to promote 
economical and efficient use of delegated author- 
ities’ revenues exceeded their authority. Indeed, 
the SCSAA states that the money that a designated 
administrative authority collects “is not public 
money” and that it “may use it to carry out activities 
in accordance with its objects or any other purpose 
reasonably related to its objects.” Although the 
Ministry may not have ownership of the revenues, 
in our view, the fact that the delegated author- 
ity’s powers to receive revenues are granted by 
provincial legislation should be sufficient authority 
for the Ministry to expect that its accountability 
arrangements adequately ensure that delegated 
authority resources are used cost-effectively and in 
the public’s interest. 

There is no authority that permits the Ministry 
to have its own auditors conduct reviews of a 
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delegated authority, and the delegated authorities 
are not subject to audits by our Office. Instead, 

the Ministry’s administrative agreements with the 
delegated authorities give it the authority to require 
a review of their operations. One such review was 
initiated for the TSSA following the 2008 propane 
explosion; another examined the TICO’s handling 
in April 2009 of the travel retailer’s abrupt closing. 
Although such reviews can be useful for addressing 
known areas of concern, they should not be a sub- 
stitute for delegated authorities establishing their 
own quality assurance programs, or for the Ministry 
routinely obtaining information on how effective 
the delegated authorities’ quality assurance pro- 
grams are. 


Delegated Authorities’ Performance 
Information Reported to the Ministry 


The Ministry’s administrative agreements with the 
delegated authorities provide for accountability 

by requiring them to report on their perform- 

ance through their business plans and annual 
reports. Examples of performance information 
required by the Ministry include information 

on compliance activities (such as the number of 
inspections, investigations, prosecutions, orders 
issued, penalties, and charges, and the amount of 
penalties assessed); processing efficiencies (such as 
turnaround times for complaints and inspections); 
the number of serious incidents; the results of client 
satisfaction surveys; and the number of complaints 
against member businesses and individuals, as well 
as against the delegated authority itself. 

We assessed the reporting requirements 
imposed on delegated authorities by the Ministry 
and noted significant improvements since our 2003 
audit. In general, the Ministry was receiving more 
relevant and useful performance information on 
a quarterly basis, and delegated authorities were 
required to explain significant variances from prior 
periods. However, we noted that inconsistencies 
exist with respect to performance information the 
Ministry requires. For example, the administrative 


agreements with OMVIC, RECO, TICO, and ESA 
provide detailed guidelines on what should be 
included in performance reporting, whereas the 
agreements with TSSA and Tarion contain very 
little detail on this subject. Figure 7 summarizes 
these reporting differences for the six delegated 
authorities we reviewed. 

In addition, none of the delegated authorities’ 
agreements required their performance measures 
to be compared with other jurisdictions and 
industry-recognized benchmarks. The ESA does 
its own limited comparisons, including comparing 
Ontario’s electrocution fatality rate per electrical 
worker to those in British Columbia, Alberta, and 
Quebec. We noted that, for example, British Colum- 
bia’s Homeowner Protection Office reports on infor- 
mation that should also be considered in Ontario, 
such as educational activities and homeowner 
awareness of consumer protection legislation; and 
the percentage of home warranty insurance claims 
that have been resolved by a builder or the Home- 
owner Protection Office’s warranty provider. 


RECOMMENDATION 8 





To better protect consumers and the public, the 
Ministry should strengthen its oversight role 
and accountability arrangements with desig- 
nated administrative authorities (delegated 
authorities) by: 

e establishing formal comprehensive account- 
ability agreements with each delegated 
authority that cover financial and oper- 
ational requirements and that would protect 
the public’s interests; . 

@ encouraging a more appropriate and fair 
balance of representation on boards of direc- 
tors between governments, consumers, the 
public, and industry; 

@ ensuring that it has the necessary authority 
over delegated authorities to access any rel- 
evant information needed, such as informa- 
tion on quality assurance programs and use 
of financial resources, that would allow for a 
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Figure 7: Inconsistencies in Requirements for Performance Information Reporting by Selected Delegated 
Authorities to the Ministry, as of March 31, 2009* 


Source of data: Ministry of Consumer Services 


Performance Information Required by the Ministry ESA OMVIC RECO Tarion TICO §=6©TSSA 


average turnaround times for processing claims to compensation 
funds or for warranties 


targets established for processing claims to compensation funds or 
for warranties 


types of inspections carried out (e.g., complaintinitiated, random, or 
targeted) - 





results of inspections (e.g., # that found deficiencies) 





# of investigations 





disciplinary fines levied against registered businesses 





key types of complaints received against businesses 


complaint-handling turnaround times and targets established for 
dealing with complaints 


disposition of complaints handled (e.g., no action required, or written 
warning issued) 


# of complaints received against the delegated authority itself oe ae 


* Figure is not inclusive of all performance information the Ministry receives from the delegated authorities, only information that is inconsistently required. 
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ee) performance information was not required to be reported by delegated authority to the Ministry 





n/a because no compensation fund is maintained by the delegated authority 





n/a because all inspections are based on claims 


Ped n/a because no fines are levied by the delegated authority against registered businesses 


comprehensive and thorough assessment of MINISTRY RESPONSE 


their financial and operational performance, a ‘ 
hg P P ‘ The delegated authorities model establishes a 
and where the Ministry’s authority to do so ei, : 
ae é a ; E framework where the Ministry retains overall 
is in question or limited, seeking the legisla- Da 

i i accountability and control of the delegated 
tive changes necessary to grant it unfettered 


Doe . legislation and regulations, and the delegated 
authority in this regard; and 
@ establishing requirements that delegated 


authorities provide consistent performance 


authorities assume the day-to-day delivery 
of their regulatory duties, including financial 
stewardship. 

The Ministry agrees with the importance of 
comprehensive accountability agreements with 


information and compare their perform- 
ance to similar organizations in other 
jurisdictions. 
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each delegated authority to ensure that the Min- 
istry has adequate tools for effective oversight. 

In 2009, the Ministry undertook an 
independent review of the delegated authorities 
model. The review concluded that the model 
is serving the public interest well. However, 
the review also identified where improvements 
could be made. 

As part of the implementation plan of the 
review’s recommendations, the Ministry will be 
renegotiating accountability agreements with 
each of the delegated authorities. The Ministry 
is reviewing pertinent oversight elements that 
should be included in the revised accountability 
agreements to make certain that public interest 
protection is enhanced, including a protocol for 
appropriate disclosure of information. 

The Ministry will work with the delegated 
authorities on a system-wide governance 
review. This will include a review of board 
composition to determine if there is a fair and 
appropriate balance of board members anda 
study of governance best practices. 

The Ministry has been working with the 
delegated authorities over the past several years 
to enhance the delegated authorities’ reporting 
of performance measures to the Ministry and, 
as the Auditor noted, has been successful in 
improving this reporting. As a part of the review 
of the delegated authorities’ accountability 
agreements, the Ministry will work with the 
delegated authorities to determine what further 
key information and performance measures 
should be included in the agreements. This 
review will include a jurisdictional comparison. 


MINISTRY PERFORMANCE MEASURES 


The Ministry’s key activities are to recommend 
changes aimed at strengthening consumer protec- 
tion legislation; to disseminate information for the 
purpose of educating and advising consumers; to 


provide information to consumers about the use 

of alternative dispute resolution techniques; to 
mediate written complaints between consumers 
and businesses; and to enforce compliance with 
consumer protection laws. The Ministry’s other 
key activity is to provide effective oversight of the 
eight delegated authorities. Although the Ministry 
is not required to publish an annual report, its 
website does include its annual Results-based Plan. 
The plan includes a description of the consumer 
protection programs and their key goals, priorities, 
and activities; performance measures the Ministry 
has established for the programs; and achieve- 
ments. No other performance measures are publicly 
reported, except for information on its website 
describing some successful prosecutions. 

The Ministry included four performance 
measures in its 2009/10 Results-based Plan. One 
measure covered its customers’ satisfaction with 
the Ministry’s handling of consumer phone inquir- 
ies and complaints; the other three performance 
measures pertained to the Ministry’s processes 
for providing oversight of delegated authorities. 

In view of the many key activities required by 

the Ministry’s mandate, we concluded that the 
performance measures reported by the Ministry 
need to be expanded if they are to allow legislators 
and the public to assess the Ministry’s performance 
and hold it accountable for the extent to which it 
achieves its mandate. 

To complement these four public measures, 
the Ministry introduced internally a set of per- 
formance measures, called a balanced scorecard, 
that included quarterly reporting on a suite of 
performance measures. The performance measures 
included turnaround times for processing registra- 
tions and issuing licences; percentage of cemetery 
owners meeting the annual filing requirements; 
number of days to resolve complaints; inspection 
hours incurred; number of investigations and pros- 
ecutions; refunds to consumers and value of can- 
celled and rescinded contracts; amounts of court 
fines and restitution; length of jail time ordered; 
and number of education events held. However, 


these additional performance measures were not 
made public. 

In our 2003 audit, we reported that the Ministry 
needed to improve the customer satisfaction survey 
about its handling of phone inquiries and com- 
plaints to ensure that the results were independ- 
ently determined and meaningful. In addition, 
there was no survey done on written complaints, 
which would be an important and perhaps more 
objective indicator of the Ministry’s effectiveness 
in dealing with consumer concerns. In this regard, 
the Ministry informed us that it successfully medi- 
ated 11% of written complaints and that 7% were 
referred to its investigations branch; about 24% 
were referred to another agency; and the remaining 
58% resulted in providing information or advice, 
or requiring no action. During our current audit, 
we noted that the Ministry had not satisfactorily 
dealt with these issues. For example, the same staff 
member who handles the phone inquiry or com- 
plaint also conducts the survey at the end of the 
call to determine whether the caller is satisfied with 
the service and the handling of the complaint, an 
approach that may not lead to an objective response 
or recording of the result. To address this concern, 
the Ministry had indicated in 2003 that it intended 
to engage an independent company to conduct the 
customer survey, but only one independent survey 
(in 2003) was conducted. No survey was done 
on written complaints either by Ministry staff or 
independently. 

In addition, the Ministry has in the past sur- 
veyed only consumers who contacted the Ministry 
with an inquiry or complaint, and not the general 
public. In our view, the kind of information indi- 
cated by our own public survey conducted by an 
independent firm on the extent to which the public 
is aware of and uses consumer protection programs 
would be important in helping the Ministry plan its 
educational activities and measure the outcomes of 
its efforts in this regard. 
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We also noted that the consumer protection 
programs in Quebec and British Columbia provide a 
wider range of publicly available performance goals 
and measures. These include the number of inspec- 
tions and investigations completed, the number 
of mediations conducted, the amounts of restitu- 
tion, and the number of licences issued. Alberta 
Consumer Protection, via Service Alberta’s annual 
report, also reports on several customer satisfaction 
measures. 

Although the Ministry reports publicly only 
limited performance information on the extent to 
which it achieves its mandate, it has been successful 
in making more detailed reporting a key require- 
ment for the eight delegated authorities that it 
oversees. 


RECOMMENDATION 9 


To improve accountability and its reporting on 

the extent to which it achieves its consumer 

protection mandate, the Ministry should: 

@ report publicly on performance targets and 
measures for all its key activities; and 

@ ona periodic basis, such as every two to 
three years, conduct independent consumer 
satisfaction surveys of its handling of both 
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telephone and written complaints. 


The Ministry agrees that appropriate perform- 
ance measures and targets should be reported 
publicly and in accordance with the require- 
ments of the new Ontario Public Service 
Directive and has committed to publishing an 
increased number of performance measures 
beginning January 2010. 

The Ministry will establish processes to 
objectively evaluate customer-service satisfac- 
tion levels as they relate to telephone and 
written complaints at the Consumer Protection 
Branch. 
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Chapter 3) Ministry of Education 
Section 


Background 





The Ontario government established the Education 
Quality and Accountability Office (EQAO) as a 
Crown agency in 1996 in response to recommenda- 
tions from its 1995 Royal Commission on Learning. 
The Commission had concluded that system-wide 
testing was necessary to monitor student achieve- 
ment and as a vehicle for assuring people that all 
students, at specific points in the learning process, 
are being assessed according to the same yardstick. 
The government also wanted to respond to the 
public’s demand for clearer information about, and 
greater accountability for, student achievement in 
Ontario’s publicly funded schools. 

The EQAO’s mandate is to develop, administer, 
mark, and report on province-wide tests of student 
achievement. Such assessment results are intended 
to provide reliable, objective, and high-quality 
data that can be used by the Ministry of Education 
(Ministry) and the province’s 72 school boards for 
student learning improvement planning. The EQAO 
is also responsible for managing and reporting on 
the province’s participation in international and 
national student testing. 

Each year, the EQAO tests students in all Ontario 
publicly funded schools in Grades 3, 6, 9, and 10. 
Grade 3 and Grade 6 students are tested in reading, 


ee'74 Education Quality and 
Accountability Office 


writing, and mathematics. Grade 9 students are 
tested only in mathematics. As a condition of high- 
school graduation, all students, including those in 
private schools, are required to pass the Ontario 
Secondary School Literacy Test (OSSLT), which is 
usually written in Grade 10. 

The EQAO develops test questions based on 
the Ministry’s school curriculum expectations. 

The questions are designed to provide an objective 
appraisal of student achievement. Tests must have 
a similar level of difficulty from one year to the next 
so that results can be compared over time. 

The EQAO provides specific guidelines for 
school boards, principals, and teachers to follow in 
delivering the tests to students. The agency then 
oversees the test scoring, usually performed by 
elementary and secondary school teachers. The 
EQAO is required to report test results to the public 
and make recommendations to the Ministry for 
improvement on any matter related to the quality or 
effectiveness of elementary and secondary educa- 
tion in Ontario. 

The assessment process is a large and complex 
undertaking, given that the EQAO must annually 
develop five different assessments in both French 
and English and then print, deliver, administer, 
collect, mark, and report on almost 600,000 tests 
given at approximately 4,300 schools. Each step in 
the process, from question development to public 
reporting, requires a number of procedures and 


controls to ensure that the results accurately reflect 
each student’s ability. 

The EQAO employs approximately 140 perma- 
nent staff complemented by as many as 1,700 
seconded and temporary staff during marking 
periods. It also relies on professional and technical 
expertise to help develop and administer the tests. 
As well, the EQAO has a Psychometric (the science 
of measuring intellectual capacity) Expert Panel 
staffed by academics from across North America 
to provide ongoing feedback on its assessment 
processes. The EQAO spent $31.7 million in the 
2008/09 fiscal year, all of it funded by the Ministry. 





Audit Objective and Scope 


The objective of our audit was to assess whether 
the Education Quality and Accountability Office 
had adequate systems, processes, and procedures in 
place to ensure that: 

@ student assessment results were comparable 
from year to year and accurately reflected 
student performance in regard to the Ontario 
curriculum; 

e legislative and policy requirements were being 
fulfilled; and 

@ goods and services were acquired and pro- 
grams delivered in an economic and efficient 
manner. . 

The scope of our audit included research on 
student assessment practices in other jurisdictions, 
reviewing and analyzing EQAO administrative 
directives, policies, and procedures, as well as inter- 
viewing agency board members and staff, including 
two psychometric experts engaged by the EQAO. 
We also interviewed personnel from four school 
boards—Peel District, Halton District, Hastings and 
Prince Edward District, and Peterborough Victoria 
Northumberland and Clarington Catholic District. 
Finally, we interviewed stakeholders such as per- 
sonnel from the Elementary Teachers’ Federation 
of Ontario, the Ontario English Catholic Teachers’ 
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Association, and the Council of Ontario Directors of 
Education. 

Our audit also included a review of the activities 
of the Ministry’s Internal Audit Services Branch, 
including its recent audit reports. We relied on their 
work in some areas to reduce the scope of our audit. 





We found that the EQAO had adequate procedures 
and controls in place to ensure that its tests accur- 


ately reflected the Ministry’s curriculum expecta- 
tions. We found that the EQAO, to ensure that the 
tests’ level of difficulty was comparable between 
years, imposed strict criteria for the development 
and field-testing of questions, that test content 
was thoroughly reviewed, and that test questions 
received multiple edits before being considered for 
inclusion in a student assessment. 

The general consensus among stakeholders, 
including principals and teachers, was that the tests 
were generally an accurate reflection of students’ 
achievement in meeting the curriculum expecta- 
tions. However, we felt that oversight of test admin- 
istration would be strengthened by ensuring that, 
over time, all school boards and schools are visited 
during test periods and that significant changes 
in year-over-year test results by school boards and 
schools are fully investigated. 

For the major areas in the testing and reporting 
process, we noted the following, including areas 
where we believe improvements can be made: 

e@ The EQAO employs a number of quality assur- 
ance measures to provide credibility to its 
processes and procedures. These measures 
help to ensure that the questions presented to 
students are appropriate for their grade level 
and represent fairly the Ministry’s curriculum 
expectations, and that the tests are consistent 
in their level of difficulty from one year to the 
next. 
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e To help monitor the administration of its tests, 


the EQAO hires an external contractor to visit 
selected schools to review pre-test prepara- 
tion, ensure test booklet security, observe the 
administration of the tests, and undertake 
other quality assurance procedures. Overall, 
the external contractor has reported a high 
degree of compliance with EQAO administra- 
tion procedures. However, although all boards 
had been visited for one or more of the assess- 
ments, an improved school selection process is 
required to reduce the risk of student cheating 
and non-compliance with administrative 
procedures. For example, 10 of the province’s 
72 school boards had not received a visit from 
the external contractor over the past five 
years to assess administration of the Ontario 
Secondary School Literacy Test (OSSLT). Yet 
the contractor visited a number of private 
schools, whose students are required to write 
the OSSLT, which had as few as five students 
taking that test. 

All students are expected to write the EQAO 
assessments, but exemptions can be granted. 
For example, principals can exempt students 
with special needs and those for whom Eng- 
lish is a second language. The public reporting 
of the overall scores on EQAO tests includes 
exempt students, but exempt students are 
counted as not having achieved the provincial 
standard in the assessment scores. Con- 
sequently, those schools with a disproportion- 
ately high number of exempt students would 
receive lower overall scores than otherwise 
comparable schools with significantly fewer 
exempt students. The teachers and principals 
we interviewed almost unanimously stated 
that this policy was unfair and could signifi- 
cantly distort reported EQAO results. 

The EQAO hires and trains as many as 1,700 
markers to grade test papers and must ensure 
consistency from one marker to the next. 
Validity papers, which are graded by an 
expert panel, are usually indistinguishable 


from regular tests and are seeded among 

the regular papers. The grades the markers 
give these validity papers are monitored to 
determine if retraining is required. The EQAO 
has a number of targets. One, which it consist- 
ently meets, is for 95% of the validity papers 
to be graded within one scoring level of the 
expert panel. However, for some questions, 
the EQAO does not always meet one of its 
other targets that 70% of the validity papers 
be graded in exact agreement with the expert 
panel—although, in recent years, the EQAO 
has moved closer to achieving this target. 

In the lower grades (Grades 3 and 6), the 
primary risk to the test’s integrity is teacher 
or principal interventions, such as coaching. 
In high school (Grade 9 and OSSLT), the 

risk shifts from the principal/teacher to the 
student—there is a higher potential for stu- 
dents to engage in collusion and other forms 
of cheating. However, the EQAO uses substan- 
tially the same quality assurance processes for 
all assessments, rather than a varied approach 
that considers the unique risks associated 
with each assessment. 

As well as examining anomalies at the student 
level, the EQAO informally reviews results at 
the school and school board levels. However, 
formal analysis and follow-up may be required 
to assess the reliability of assessment results. 
For example, we noted that some schools’ 
EQAO results fluctuated by as much as 50% 
from one year to the next, but these instances 
were not being systematically flagged for 
follow-up to determine what accounted for 
such a dramatic change. 

To help motivate applied math students who 
have consistently fallen short of the provincial 
standard, schools are allowed to incorporate 
the EQAO scores in student report card marks 
for Grade 9 math. However, we found that 
this was not consistently done throughout the 
province because EQAO scores accounted for 
anywhere from zero to 15% of a student’s final 
mark. 


e The EQAO annually reports on student testing 
results as well as the results from question- 
naires on its activities given to students, teach- 
ers, and principals. However, the school staff 
we interviewed stated that the questionnaires 
did not sufficiently allow for feedback on ways 
to improve the testing process. Also, they 
agreed generally that the EQAO should take a 
bigger role in explaining to parents and other 
stakeholders the assessment process and 
how it can promote improvement in student 
learning. 

e@ The major expenditures for the EQAO relate 
to the administration of student assessments, 
such as staffing and the hiring of tempor- 
ary test markers, as well as test printing, 
warehousing, and delivery. We found that 
the EQAO had developed a good budgeting 
process to help control costs and had reduced 
its annual expenditures by over 20% during 
the past five years while delivering substan- 
tially the same service. The EQAO must follow 
Management Board of Cabinet Directives in 
the acquisition of goods and services, and 
we found that it complies with the required 
tendering practices and that the necessary 
procurement documentation and approvals 
were on file. In addition, the Ministry’s Audit 
Services Team found that the travel expense 
procedures maintained by EQAO were, over- 
all, operating effectively. 


The EQAO plays an important role in Ontario’s 
education system. The agency provides an 
independent check on all students at specific 
points in their learning, a measure of the qual- 
ity and accountability of our publicly funded 
schools and important information for student, 
school, and system-wide improvements. 

We are pleased that the Auditor General’s 
audit attests to the rigour of the assessment 
practices and processes at the core of the 
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EQAO’s work. The audit confirmed that the tests 
are an accurate reflection of the Ministry of Edu- 
cation’s curriculum expectations, that their level 
of difficulty is comparable between years, and 
that the administration and marking processes 
ensure that results are valid, consistent, and a 
reliable indication of student achievement. We 
are also pleased with the finding that stakehold- 
ers are in agreement that the tests reflect the 
provincial curriculum expectations. 

The EQAO is proud that the report confirms 
the agency’s solid financial practices and 
acknowledges the reduction in annual expendi- 
tures by over 20% in the past five years, while 
delivering substantially the same service. In 
keeping with our commitment to continuous 
improvement, we welcome the Auditor Gen- 
eral’s recommendations and will closely con- 
sider each one in order to further strengthen the 
assessment program. As we do so, we will give 
particular attention to better explaining and 
promoting the assessment program to parents 
and the general public. 


Detailed Audit Observations 





OVERVIEW OF EQAO TESTING 


Since the 2000/01 school year, the EQAO has 
administered approximately 600,000 tests annu- 
ally. Figure 1 illustrates the test breakdown for 

the 2008/09 school year. The Grade 3 assessment 
began in 1997, followed by the Grade 6 assessment 
in 1999, and the first assessment of Grade 9 math- 
ematics in 2001. In October 2000, the EQAO admin- 
istered the first trial Ontario Secondary School 
Literacy Test, required for high-school graduation. 
The English and French version of all tests have 
the same number and type of questions, but reflect 
variations in the curriculum for the two languages. 


Chapter 3 * VFM Section 3.04 





132 4 2009 Annual Report of the Office of the Auditor General of Ontario 


With the exception of the OSSLT, where the stu- 
dent either passes or does not, the tests are scored 


students score at least at a Level 3 standard, equiva- 
lent to a B average, on province-wide EQAO testing 
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on a four-point scale with four being the highest for reading, writing, and mathematics. 
Province-wide results for all tests since 


1999/2000 are shown in Figures 2, 3, and 4. 


mark and roughly equivalent to an A grade. The 
Ministry’s overall goal is to have 75% of 12-year-old 


Figure 1: EQAO Testing — 2008/09 
Source of data: EQAO 
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to ensure that the number and types of questions, 








Figure 2: Grades 3 and 6 — Percentage of Students Achieving Provincial Standard (Levels 3 and 4) — 
1999/2000-2008/09 


Source of data: EQAO 














Grade 3 Grade 6 
School Year ing M ad he 
1999/2000 49 52 oF 45 50 44 50 48 51 58 58 5/7 
2000/01 49 D2 61 41 5 40 55 53 54 54 57 60 
2001/02 50 55 58 44 55 47 55 53 54 58 61 63 
2002/03 50 55 ay 47 58 47 56 54 53 58 63 66 
2003/04 54 58 64 49 63 55 58 54 57 63 68 70 
2004/05 59 61 66 49 68 ay 63 59 60 67 70 74 
2005/06 62 64 68 56 72 59 64 61 61 68 % 76 
2006/07 62 64 69 54 73 61 64 61 59 68 74 76 
2007/08 61 66 68 60 74 62 66 67 61 US 80 78 
2008/09 61 68 70 66 76 66 69 67 Bias 77 79 80 


SS 
Figure 3: Grade 9 Mathematics — Percentage of 


Students Achieving Provincial Standard 


(Levels 3 and 4) — 2000/01-2008/09 
Source of data: EQAO 






English-speaking French-speaking 


School Year 

2000/01 49 13 45 10 
2001/02 ; 64 24 65 22 
2002/03 66 21 66 20 
2003/04 68 26 68 2s. 
2004/05 68 27 69 24 
2005/06 i 30 70 eV 
2006/07 rp 35 70 23 
2007/08 ie 34 67 34 
2008/09 77 38 68 40 





the coverage of the Ontario curriculum, and the level 
of difficulty are comparable from year to year. 

The EQAO recruits and trains educators with 
expertise in literacy and mathematics to apply the 
blueprints on “item-writing” committees that devise 
the test questions. The EQAO provides committee 
members with its Development Specification Guide 
to assist them in drafting possible questions for 
future EQAO tests. 

The EQAO employs several different quality 
assurance processes to ensure that the questions 
are appropriate before they are included in a formal 
EQAO test. For example, designated teachers could 
give some proposed questions to students and, 
based on the results, modify or eliminate them from 
the bank of draft questions. Another field-testing 
process is to include proposed questions that are 
indistinguishable from actual EQAO questions 
for possible inclusion in subsequent assessments. 
Although the answers to these questions would 
not be part of a student’s formal score, the overall 
results would be used to maintain consistency in 
the difficulty level of questions from year to year. 

Before a question is included in an EQAO test, 
it is reviewed by education professionals on two 
EQAO committees. The Assessment Development 
Committee ensures questions are based on the 
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Ne ee ee 
Figure 4: Grade 10 — Percentage of Students who 


Passed the Ontario Secondary School Literacy Test — 
2000/01-2008/09 


Source of data: EQAO 





School Year English-speaking French-speaking 
2000/01 68 54 
2001/02 75 ae OL 
2002/03 72 79 
2003/04 TT Geer ey iars 
2004/05 82 Fees 80 
2005/06 84 fae eer 81 
2006/07 84 a 83 
2007/08 84 - 83 
2008/09 85 eee BA 


Ontario curriculum. The Sensitivity Committee 
ensures that the questions are culturally fair to the 
broadest range of students and are free of any bias 
based on factors such as gender or race. 

Overall, we found that the EQAO imposed 
strict criteria for the development and field-testing 
of questions, that test content was thoroughly 
reviewed, and that any question received multiple 
edits before being considered for inclusion in an 
EQAO assessment. 

To further enhance the credibility of its tests, 
the EQAO has created a psychometric expert panel 
composed of seven university professors and experts 
from different organizations across Canada and 
the United States. The panel semi-annually reviews 
EQAO procedures and provides recommendations 
to improve its assessment-development process. 

Several teachers and school principals who we 
interviewed expressed general satisfaction with the 
EQAO tests. However, some expressed concerns 
in regard to the complexity of some questions and 
improving cultural/socio-economic sensitivity. 

For example, one teacher stated that the school’s 
students had difficulty with a question about a 
menu because the school was in a very low-income 
community where few of the children went to 
restaurants. 
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We also interviewed two of the psychometric 
experts who provide ongoing advice to the EQAO 
on matters related to testing models and the more 
technical aspects of assessments, such as Item 
Response Theory, which provides a framework for 
evaluating how well an assessment works and how 
well it measures student achievement by allowing 
comparisons of assessment results over time. Both 
experts agreed that the EQAO process is thorough 
and ensures consistency from one year to the next. 

Also, based on our interviews with a number 
of stakeholders, including the teacher federations, 
we found that despite philosophical concerns with 
universal testing, they voiced general satisfaction 
with the test-development process and agreement 
that EQAO tests reflected provincial curriculum 
expectations. 


Administration of EQAO Testing 


EQAO assessments are administered in thousands 
of schools across the province at scheduled times 
during the school year. By necessity, the EQAO 
counts on the co-operation and professionalism of 
school principals and teachers to administer the 
assessments in accordance with its guidelines. Each 
guideline contains the procedures to be followed by 
assessment administrators. 

School principals are responsible for ensuring 
that teachers are prepared to administer the test, 
that the test administration process is well organ- 
ized, and that all eligible students write the assess- 
ments. Principals also must ensure the security of 
test booklets before and after the tests and that all 
are collected and returned to the EQAO. 

It is expected that students will work independ- 
ently to solve questions and write their responses 
during the assessment. Teachers must not say or 
influence student responses or encourage students 
to alter their responses. Any circumstance that 
could affect the validity of student performance is 
to be reported promptly to the EQAO. 

Based on our interviews at a number of primary 
and secondary schools, we found that procedures 


existed to maintain the security of all test materials 
and that test administrators had received adequate 
training. Teachers and principals commented that 
the EQAO’s call centre was very helpful and that 
EQAO staff responded to concerns in a timely 
fashion. 

We also found general satisfaction with all 
of the test administration guides. Teachers and 
principals commented that the EQAO had made 
significant improvements to the guides in recent 
years, increasing clarity and ease of implementing 
all requirements. The teachers’ only common con- 
cern was a desire to see all significant changes from 
previous guides bolded, highlighted, or otherwise 
communicated in a way to ensure that nothing of 
consequence is missed. 

To monitor whether EQAO guidelines are fol- 
lowed and to reduce the risk of improprieties, 
the EQAO has hired an external contractor to 
send quality assurance monitors to visit selected 
schools at the time of testing. The monitors review 
pre-test preparation, ensure test booklet security, 
observe the actual administration of the tests, and 
undertake other quality assurance procedures to 
ensure that the schools are following the EQAO’s 
requirements. Although the majority of schools 
are selected at random, some schools are visited 
because of concerns expressed regarding the prior 
year’s test administration. In the past five years, 
monitors have visited over 1,300 schools or about 
260 schools annually. Overall, the external con- 
tractor has reported a high degree of compliance 
with EQAO administration procedures. 

However, although all boards had been visited 
for one or more of the assessments, we found 
that 10 of the province’s 72 school boards had not 
received a visit from a quality assurance monitor 
to check, for example, the OSSLT during those five 
years. One of the boards that had not received a 
visit during OSSLT assessments had over 20 high 
schools. In addition, we noted that 14 of the 30 
private schools the contractor visited had fewer 
than 20 eligible students—some with as few as 
five students—writing the OSSLT test. Meanwhile, 


other private schools with more than 100 eligible 
students were not visited. 


Assessment Exemptions and 
Accommodations 


All students are expected to write the EQAO assess- 
ments, but principals can grant exemptions to 
students with special needs and to those attending 
English schools where that is their second lan- 
guage. However, many students with special needs 
can demonstrate their level of competence in EQAO 
tests with the special accommodations that they 
would normally receive in school. For example, vis- 
ually impaired students can be given EQAO tests in 
a Braille format and those with learning disabilities 
may be able to complete answers if given more time 
than is normally allotted. 

In Ontario, a school receives no advantage by 
exempting students from EQAO assessments. The 
exempted students are assessed as not achieving 
the provincial standard and are included ina 
school’s overall results. Consequently, it is to the 
school’s advantage to encourage students to write 
the tests. However, the teachers and principals we 
interviewed almost unanimously stated that this 
policy was unfair because schools with a dispropor- 
tionate number of exempt students—some schools 
have exempted as many as 10% of their students 
from EQAO testing—could significantly distort its 
overall EQAO results. 

We reviewed the number of EQAO assessment 
exemptions granted over the past five years and 
noted an almost 40% decline in the number of 
Grade 3 and Grade 6 exempted students. For 
example, exemptions for the Grade 3 writing 
assessment fell from 8,100 in the 2003/04 school 
year to 4,800 students in the 2007/08 school year. 
This trend has resulted from a number of school 
boards making concerted efforts to ensure that as 
many students as possible write the tests. However, 
we noted several boards where the number of 
exempt students has remained relatively constant 
year over year, or even increased. 
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RECOMMENDATION 1 


To improve the Education Quality and Account- 
ability Office’s (EQAO’s) test development 

and administration process and to ensure that 

student assessments continue to be reliable and 

objective and that all students are given the 
opportunity to demonstrate their competence, 
the EQAO should: 

e highlight to principals and teachers any sig- 
nificant changes in the compliance require- 
ments outlined in the guides to administer 
EQAO testing; 

@ improve the process for selecting the schools 
visited by quality assurance monitors to 
ensure that all school boards and large pri- 
vate schools are periodically monitored; 

@ assess the equity of including exempt stu- 
dents in the overall assessment results as 
having not met the provincial standard; and . 

e identify schools and school boards where the 
number of exempt students appears to be 
relatively high and follow up to ensure that 
exemptions are justified. 


The EQAO is pleased that educators have 
recognized the improvements to the administra- 
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tion guides in recent years and that they feel 
these changes have increased clarity and ease 
of implementing the requirements. The EQAO 
agrees that significant changes year to year 
should be highlighted in the administration 
guides. 

The EQAO agrees with the recommendation 
to introduce additional elements to the process 
for selecting schools to be visited by quality 
assurance monitors. Currently, in the random 
selection of schools for quality assurance visits, 
schools are stratified to ensure proportional rep- 
resentation across the six regional districts and 
by type of school (public, Catholic, and private). 
The EQAO will ensure representation across 
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school boards for the upcoming assessment in 
2009/10. 

The policy of accounting for every student 
reflects the overarching principle that Ontario 
schools are responsible for the achievement of 
all students attending their schools. Principals 
make the determination, together with parents, 
about which students are unable to write the 
assessment even with accommodations or 
special provisions. If the EQAO were to exclude 
exempted students when reporting a school’s 
results, those schools that ensure that all stu- 
dents are included would consider the practice 
inequitable should other schools not have the 
same approach. It is important that all students 
have the opportunity to demonstrate their 
achievements. The current practice provides for 
valuable insights into all students’ learning and 
it supports accountability for student and school 
performance. In addition to results for all stu- 
dents, the EQAO does provide separate reports 
for participating students in each school’s public 
report. Both sets of results are valid and provide 
different information. 

The agency agrees that it should follow up 
with school boards and schools where exemp- 
tion rates remain high. It is important to rec- 
ognize that there are some schools where high 
exemption rates are appropriate due to specific 
student populations, such as specialized schools 
or classes within schools that service children 
with multiple disabilities. Where this is not the 
case, the EQAO will take appropriate action. 


ASSESSMENT MARKING AND ANALYSIS 
Marking of EQAO Assessments 


School principals are responsible for collecting all 
completed test papers, sealing them in bins that are 
bar-coded, and shipping them to the EQAO. The 
papers arrive at the marking area in the sealed bins 
to ensure they have not been tampered with in tran- 


sit. The EQAO rents space at a large convention cen- 
tre to house as many as 1,700 markers to grade the 
test papers. The marking for all Grade 3, 6, and 9 
assessments takes place in the summer. As a result, 
the EQAO is able to hire qualified elementary and 
secondary school teachers as markers. For the 
OSSLT, which is graded in the spring of the school 
year, although many of the markers are qualified 
current or retired teachers, the EQAO hires mark- 
ers who have a required minimum education level 
equivalent to an undergraduate university degree. 

EQAO tests consist of closed-response (multiple 
choice) and open-response (written answers) ques- 
tions. The closed-response answers are machine 
read, eliminating any human variability in marking. 
However, the marking of open-response questions 
is more subjective because two markers may have 
different opinions on an appropriate grade. 

In an attempt to heighten consistency, the 
EQAO has established several different quality 
control procedures. These procedures begin with 
a framework termed a Quality Management Plan 
designed to ensure that the marking process is 
run efficiently and effectively. The plan, updated 
annually, includes the process for recruiting mark- 
ers, outlines their training, and schedules the daily 
activities required to ensure quality grading. It also 
deals with ongoing supervisory review and signoff 
procedures. 

At the beginning of the marking process, all 
markers are provided training to develop a com- 
mon understanding for interpreting and applying 
the requirements. Markers are trained to grade 
only one question, using anchor papers that give 
examples of answers at various grade levels and a 
scoring rubric that describes what is expected from 
student answers. Finally, assessment markers are 
required to pass a qualifying test. 

Over and above these safeguards and proced- 
ures, a consultant hired by the EQAO in 2004 
recommended that on-line training would help 
maintain more consistent standards because all 
markers would receive identical instructions. In 
addition, on-line training would allow markers to 


set their own pace and to train at home prior to the 
start of the marking process. We also noted that 
on-line training could improve productivity and 
provide timelier feedback to assessment markers. 
However, the EQAO has not implemented this 
procedure. 

The EQAO uses “validity papers” to monitor 
each marker’s accuracy. Validity papers are pre- 
marked by experts and normally circulated uniden- 
tified throughout the marking session to enable 
monitoring on both a daily and cumulative basis. 
The purpose is to determine if markers are grading 
questions according to standards established by the 
expert panel and if retraining is required. 

The EQAO has established validity targets to 
be achieved in the marking process. One target is 
for 95% of the validity papers to be marked either 
in exact agreement or within one scoring level of 
the expert panel’s assessment. For the 2008 Grade 
3, 6, and 9 assessments and for the 2009 OSSLT, 
the EQAO met its 95% validity target for nearly all 
questions marked. 

However, for some questions, the EQAO does 
not always meet one of its other targets that 70% of 
the validity papers be marked in exact agreement 
with the expert panel—although, in recent years, 
the EQAO has moved closer to achieving this target. 

Another aspect of meeting validity targets is to 
ensure that markers grade a sufficient number of 
validity papers during the marking process. For the 
2008 assessments, we noted that markers were not 
marking enough validity papers early in the process 
to identify those who did not meet the required 
accuracy targets for potential retraining. In 2009, 
for the OSSLT, the EQAO increased the validity 
reads per marker, but the number varied dramatic- 
ally as some markers graded more than 150 validity 
papers while others graded fewer than 40. 

We noted that a process called backreading is 
employed in other jurisdictions whereby super- 
visors read a certain percentage of the papers that 
have already been graded in order to focus on 
the work of markers who are not meeting validity 
targets. Supervisors can intervene, retrain, or even 
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dismiss markers who fail to grade papers accurately 
and consistently. 

Overall, we concluded that, although there is 
room for improvement, the assessment marking 
process is sufficiently controlled to ensure that 
results are valid, consistent, and reliable. 


RECOMMENDATION 2 


To improve the assessment marking process to 
ensure that results continue to be valid, consist- 
ent, and reliable, the Education Quality and 
Accountability Office should: 

@ consider adopting on-line training for assess- 
ment markers; 

@ examine different methods to increase the 
number of validity reads for each marker, 
especially early in the marking process; and 

@ consider implementing supervisory back- 
reading to help improve marker accuracy. 

The EQAO continually looks for ways to 
enhance its scorer processes. The 2004 recom- 
mendation regarding on-line training was in the 
context of an image-based scoring approach. 

Image-based scoring was thoroughly considered 

but deemed not to be appropriate at that time. 
The EQAO continues to explore technology 
solutions that will address the requirements of 
the EQAO’s Ontario-based program, including 
various approaches to on-line training. 

The EQAO agrees with the recommendation 
regarding validity reads and introduced improve- 
ments to the process in 2009. The changes to the 
process resulted in an increase in the number of 
validity papers scored by each scorer. The EQAO 
sets its validity targets for each item on an assess- 
ment to ensure accurate and reliable student 
achievement results. In 2009, 95% of validity 
papers were either in exact agreement with the 
expert panel’s score or within one scoring level 
of the expert panel’s score. The EQAO also sets a 
validity target for exact agreement (same score) 
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as the expert panel’s score. This is a performance 
target that the EQAO has established as a best 
practice and, through various process improve- 
ments, has improved over the past five years. We 
are the only jurisdiction that sets this target and 
publicly reports against it. 

The EQAO agrees that backreading is one 
of many scoring procedures that can be used — 
to ensure validity and reliability of scoring. 
Jurisdictions normally choose either the process 
of validity paper insertion or backreading. The 
EQAO, in consultation with its Psychometric 
Expert Panel, will consider backreading as a pos- 
sible additional measure to monitor and support 
scorers identified as requiring further training. 


Assessment Analysis and Follow-up 


For some parents, EQAO test results are the only 
public information they have to assess local school 
and school board performance. In the 2007/08 
school year, about 1,000 out of the 3,500 elemen- 
tary schools had less than 50% of their students 
attain Level 3, the provincial standard, in Grade 3 
reading. The top performing board in the province 
had 73% of its students achieve Level 3 or better; 
the lowest performing board stood at 49%. 

Many teachers and principals commented that as 
EQAO results take on broader acceptance, there is 
ever-increasing pressure to improve results that form 
the basis of ministry and school board interventions 
and private organization rankings. In fact, others 
often rely on rankings for non-education purposes. 
For example, real estate agents use them to attract 
parents to areas with high performing schools. 

To ensure that reported results are valid, reli- 
able, and accurate, the EQAO employs several 
different quality assurance procedures. Such 
procedures include integrity software to identify 
unusual school response patterns in multiple choice 
questions that would suggest collusion among stu- 
dents, the review of some open-response answers 
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from 5% of the schools to determine if there are 
any patterns that indicate collusion, and the inves- 


_ tigation of complaints to determine if there is any 


evidence of impropriety. Although the EQAO does 
not have a formal complaints process, the majority 
of the 14 investigations in 2007/08 arose from con- 
cerns expressed by principals, teachers, and school 
board staff. 

In regard to potential cheating by its students, 
British Columbia has a formal complaints process 
that outlines the responsibilities of students, 
schools, and school boards with standardized forms 
that are to be completed to describe each incident 
and what actions were taken. 

If the EQAO identifies a problem with the 
results of a school or school board, the results are 
not reported publicly. In 2006, for example, four 
elementary schools in different boards had their 
results withheld due to an investigation that deter- 
mined that the students had been inappropriately 
coached. In 2007, on the basis of complaints from 
staff, all 24 schools in one board had their results 
withheld because test materials were inadvertently 
distributed by the board office. In 2008, no school 
results were withheld. 

There is no public disclosure of why results 
are withheld, although more complete and open 
communication would no doubt act as a deter- 
rent to help ensure compliance with assessment 
guidelines. The teachers and principals we spoke 
to indicated that they were unaware of the results- 
suppression policy. They suggested that the 
administration guide should be more explicit on 
the repercussions of policy violations. We noted 
that the Massachusetts administration guide is very 
clear and explicit on penalties, such as the loss of 
a teaching licence, for violating the assessment 
administration guide. 

Based on a review of past concerns and discus- 
sions with EQAO staff, we noted in the lower grades 
(Grades 3 and 6) that the primary risk to the test’s 
integrity is non-compliance with administrative 
procedures by teachers and principals. In high 
school (Grade 9 and OSSLT), the risk shifts from 


the principal/teacher to the student where there 

is a higher potential for students to engage in 
collusion and other forms of cheating. However, 
although there are some variations in the OSSLT 
quality-assurance process, the EQAO uses substan- 
tially the same processes for all assessments rather 
than a varied approach that considers the unique 
risks associated with each assessment. 

As well as examining anomalies at the student 
level, the EQAO informally reviews results at the 
school and school board levels. However, more 
formal analysis and follow-up may be required 
to ensure that the testing process is effective in 
improving student performance. For example, some 
EQAO board members expressed concern that over 
the past three years only about 35% of Grade 9 
applied mathematics students achieved Level 3 on 
the EQAO assessment. Several wanted the EQAO to 
formally investigate whether these students were 
not motivated to write the tests, received inad- 
equate instruction, or whether there were problems 
with the curriculum. 

In an attempt to motivate Grade 9 students, 
the EQAO has allowed school boards across the 
province to incorporate Grade 9 EQAO results into 
a student’s end-of-course mark. Ministry policy 
states that the end-of-course exam can count for 
up to 30% of a student’s final mark. In the 2008/09 
school year, one of the school boards we visited 
decided that EQAO results would count for 15% of 
the student’s final Grade 9 math mark. The degree 
to which these test results form part of the final 
mark is inconsistent province-wide, ranging from 
zero to 15%. 

In our audit, we found significant variations in 
year-over-year assessment results. For example, we 
found that some 10% of the schools’ Grade 3 results 
over the past four years decreased by more than 
20%, while another 10% increased by more than 
20%. This demonstrates that although there may be 
a gradual overall upward trend in provincial EQAO 
results, there can be significant fluctuations at the 
school level that may warrant following up to assess 
whether the changes are reasonable. 
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For some schools, the change in assessment 
results was greater than 50% from one year to next. 
For example, in one school, fewer than 40% of 
Grade 3 students achieved the provincial standard 
from 2004 to 2007, but its results increased to 
100% in 2008. Although one might question the 
reasonableness of such a dramatic improvement, 
such significant swings could be caused by many 
legitimate factors, such as a different cohort of stu- 
dents, a change in staff, improvements initiated by 
the Ministry, or interventions by the school board 
or school to improve results. 

The EQAO does not undertake a formal analysis 
or investigation to determine the cause of sudden 
and significant changes to ensure their legitimacy. 
Such analysis could identify whether significant 
variations are justified or result from the testing 
process and are areas the EQAO should consider for 
intervention. 


RECOMMENDATION 3 


To ensure that assessment results continue to 

be reliable, consistent, and valid, the Education 
Quality and Accountability Office (EQAO) should 
enhance its quality assurance procedures by: 

@ implementing a formal complaints process to 
help determine if there are any trends and to 
identify potential actions that could prevent 
non-compliance with assessment guidelines 
or student cheating; 

e considering more complete disclosure when 
test results at a particular school are with- 
held as a deterrent against non-compliance 
with assessment guidelines; 

@ outlining in its administration guides poten- 
tial penalties for violating EQAO policy; 

@ tailoring its quality assurance processes to 
address unique risks associated with differ- 
ent assessments; 

@ reviewing Grade 9 applied mathematics 
results to assess whether incorporating 
EQAO results into the student’s final mark 
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is effective in motivating students and, if so, 
suggest a more consistent approach; and 

@ investigating any abnormally large varia- 
tions in school assessment results from year 
to year and ensuring that they are justified. 





The EQAO takes complaints regarding non- 


compliance of assessment guidelines very 
seriously and has rigorous quality assurance 
processes to ensure that the administration of 
the assessment is consistent across the province. 
The EQAO has always followed up complaints 
at the school and board level and in 2009 intro- 
duced a standardized format for investigating at 
the school and board level. The EQAO now has a 
clear protocol for investigating and withholding 
results when warranted and will continue to 
review approaches in other jurisdictions. 

The EQAO agrees with the recommendation 
that in instances where non-compliance with 
assessment administration guidelines have been 
confirmed, it should disclose the reason that 
results are being withheld. The EQAO will also 
outline potential consequences for non-compli- 
ance with its administrative guidelines. 

The EQAO applies all quality assurance 
procedures to all assessments because it is 
important to consider the same elements, such 
as proper administrative procedures, security 
of materials, principal and teacher compli- 
ance, and student cheating. However, for some 
assessments, certain procedures are used more 
extensively, such as when comparing student 
results for collusion in Grade 9 math and the 
OSSLT. The EQAO will continue to examine its 
quality assurance processes and tailor specific 
strategies to meet the varying conditions of the 
assessments. 

The EQAO agrees with the recommendation 
to review the practice of applying EQAO results 
to Grade 9 math school results. In 2010, it will 


include on the Grade 9 teacher questionnaire 
questions about the practice of counting EQAO 
results for course marks and will then correl- 
ate this information with student achievement 
results to determine the best course of action. 
The EQAO has always had a practice 
of reviewing significant changes in school 
results and contacting directors of education 
for those schools with such changes. A more 
formal analysis was introduced in summer 
2009, whereby superintendents responsible 
for schools identified with large variations are 
contacted and asked to conduct a review and 
provide a written report outlining explanations 
for any large gains. As noted in the Auditor Gen- 
eral’s report, significant swings could be caused 
by many legitimate factors. In the review of such 
schools to-date, those with large variations have 
been attributed to such factors. 


Reporting on EQAO Assessment Results 


The EQAO is required by legislation to report to 
the public and to the Minister of Education on the 
results of its testing and, generally, on the quality 
and effectiveness of elementary and secondary 
school education. For both English and French 
language students, EQAO assessment results for 
each subject area are reported by school, school 
board, and on a province-wide basis. These results 
are compared to prior years and are also reported 
by gender, by English-language learners, and by 
special needs students. More detailed contextual 
results are available to schools and school boards 
through a secure website. Parents also receive an 
individual student report detailing their child’s 
results. 

In addition to the statistical data reported annu- 
ally, the EQAO also provides a series of reports 
that include a summary of high-level trends, 
school success stories, and strategies for student 
improvement. These reports provide principals and 


teachers insight into areas that may need attention, 
and the principals/teachers we spoke to indicated 
that EQAO results helped them plan strategies to 
enhance classroom learning. Teachers also com- 
mented that the EQAO gave them feedback on how 
well they conformed to the curriculum. 

These EQAO annual provincial reports also 
include the results of questionnaires filled out by 
students, principals, and teachers. For example, the 
teachers’ questionnaire asks whether they make use 
of EQAO data and other specific resources such as 
the school library and computer software. Many of 
the teachers we interviewed stated that the ques- 
tionnaires were repetitive from year to year and did 
not allow for general feedback or the opportunity 
to raise other issues. In 2009, the EQAO initiated 
a pilot communications strategy to obtain more 
open feedback from a number of school staff on the 
EQAO’s student assessment process. 

In 2007, the EQAO expanded its outreach 
program to provide, on request, workshops and 
seminars to assist school board and school staff in 
understanding and using EQAO data to improve 
student achievement. EQAO outreach staff conduct 
regional workshops that all boards can attend and 
visit boards individually. Some boards have been 
visited as many as 10 times, while other school 
boards across the province have received relatively 
few visits from outreach staff. 

In January 2009, the EQAO Board of Direc- 
tors requested that its management initiate a 
bolder communications strategy with the public 
and school communities. In particular, the board 
wanted parents and the general public to better 
understand the benefits of the assessment process 
to promote improvements in student learning. 
Several of the school principals and teachers we 
interviewed agreed that the EQAO should take a 
larger role in explaining and promoting the assess- 
ment process. 
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RECOMMENDATION 4 


To further improve its policies and processes and 
the procedures designed to produce accurate 
and reliable reports that can be used to improve 
student performance, the Education Quality and 
Accountability Office (EQAO) should: 

@ consider formalizing its pilot initiative to 
provide more open-ended questions for 
principals, teachers, and students to obtain 
better feedback on any concerns with the 
assessment process and ways to improve it; 

e develop a more formal outreach strategy 
to give all schools and school boards an 
opportunity to gain further insight into the 
value of EQAO data and how it can be used 
to improve student learning; and 

@ increase the understanding of parents and 
the general public of how the assessment 
process enhances student learning. 





It is important to ask the same questionnaire 


questions each year so that comparisons over 
time can be made. The EQAO has conducted 
research on factors that are related to student 
achievement and will be revising its question- 
naires to gather data on these factors. Revised 
questionnaires will be implemented in 2010 for 
the primary and junior assessments. The EQAO 
recognizes the value of gathering feedback from 
educators and does this through a variety of 
outreach techniques, including focus groups, 
educator feedback on the public web site, the 
EQAO WebMag, monitor visits by the Council 
of Ontario Directors of Education, and OSSLT 
shadow activities. 

The EQAO’s outreach program was estab- 
lished with the goal of helping all schools and 
school boards understand the value of EQAO 
data and how best to use this information in 
their improvement-planning processes. The 
EQAO agrees with the recommendation and will 
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continue to enhance its outreach activities to 
ensure that schools and boards are able to use 
this valuable data. 

The EQAO agrees with the recommendation 
to increase understanding of how the assess- 
ment process enhances student learning. It will 
continue to enhance its public reporting practi- 
ces and communications materials for parents. 
The EQAO Board of Directors established this as 
a priority in its 2009/10 business plan. 


ADMINISTRATION COSTS 


The major expenditures for the EQAO relate to 
the administration of student assessments such as 
staffing and the hiring of temporary test markers, 


as well as test-printing, warehousing, and delivery. 


The EQAO spent almost $32 million to deliver its 
services during the 2008/09 fiscal year. We found 


that the EQAO had developed a good budgeting 
process to help control costs and had reduced its 
annual expenditures by over 20% during the past 
five years while delivering substantially the same 
service. Major cost reductions were achieved in 
the assessment marking process, print production, 
warehousing, and test distribution. 

The EQAO is obliged to follow Management 
Board of Cabinet Directives in the acquisition of 
goods and services, and we found that it complies 
with the required tendering practice and that the 
necessary procurement documentation and approv- 
als were on file. In addition, in 2008, the Ministry’s 
Audit Services Team reviewed selected EQAO 
financial processes and found that its travel expense 
procedures were operating effectively, overall, and 
that all expenses were supported prior to payment in 
accordance with the Management Board of Cabinet’s 
Travel, Meal, and Hospitality Expense Directive. 
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Background 


In the 2008/09 fiscal year, Ontario ministries 

and agencies collected almost $2.2 billion in 
revenues that are classified as Fees, Licences, and 
Permits—hereafter referred to as user fees—and 
reported in Other Revenue in the Public Accounts 
of the province. Revenues from user fees represent 
about 2% of total annual provincial revenues. Of 
the rest, about 69% of Ontario’s revenues comes 
from taxation; 18% from transfers from the federal 
government; and the remaining 11% from other 
sources such as sales, rentals, royalties, and fines. 
The difference between a user fee and a tax is that a 
user fee is generally charged to recover all or a part 
of the costs of providing a specific good or service 
to the individuals and businesses that request it, 
such as a driver’s licence; a tax is used to produce 
revenues for general government purposes and for 
goods and services that the government deems to 
be a “public good”—available to all individuals but 
paid for by the public as a collective entity, such as 
health care, the court system, and education. 

Over 400 types of user fees are charged to indi- 
viduals and businesses by ministries and agencies, 
such as for registration and search services and for 
the issuing of licences. The Ministry of Transporta- 
tion collects almost half of all user-fee revenues— 





Alcohol and Gaming Commission of Ontario and Ministries of 
Environment, Finance, Government Services, and Transportation 


emily Government User Fees 


for vehicle registration, carrier, and driver licence 
fees. The Alcohol and Gaming Commission of 
Ontario collects another 22%—for liquor licences 
and permits. Figure 1 shows user fees charged in 
2008/09 by activity and ministry or agency. 

Under the Financial Administration Act, all 
ministries and certain agencies are required to 
deposit any revenues, including user fees, into the 
province’s Consolidated Revenue Fund to be used 
for general government purposes. In other words, 
these revenues are not earmarked for particular 
programs or restricted in their use. Exceptions exist 
for fees that are deposited in the Consolidated Rev- 
enue Fund but are designated for special purposes 
under legislation. For example, the Ministry of Nat- 
ural Resources collects approximately $116 million 
per year relating to provincial parks and fish and 
wildlife management. 


Audit Objective and Scope 


Our audit objective was to assess whether selected 
ministries had adequate systems, policies, and pro- 
cedures in place for government user fees to ensure 
that: 
e fees were properly justified and were 
authorized, administered, and reported in 


143 


rs) 
eS 
© 
= 
S 
3 
© 
® 
~” 
= 
ra 
= 
e 
” 
the 
2 
2. 
© 
= 
o 





144 J 2009 Annual Report of the Office of the Auditor General of Ontario 


ee 


Figure 1: Revenues from User Fees, 2008/09 ($ million) 
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Source of data: Public Accounts of Ontario 


other - various ministries and agencies ($490.8) 


vehicle and carrier registrations and driver's licences - 
Ministry of Transportation ($1,033.9) 






company registrations - Ministry of Government Services ca cna : 


gaming registrations and lottery event licences - 
Alcohol and Gaming Commission ($32.1) 








Drive Clean program - Ministry of the Enviornment Ci aa <a _—?£“ 


personal property registrations - Ministry of Government Services ($43. 


estate and court-+elated fees - Ministry of the Attorney General ($55.1) 


liquor licences and permits - Alcohol and Gaming Commission ($468.0) 


compliance with government and legislated 
requirements; 

® proper controls existed over fee collection and 

adequate quality-of-service standards existed 
for services with fees; and 

@ fees were periodically assessed and reported 

on to ensure that they met established 
requirements. 

We selected the ministries of the Environment, 
Government Services, and Transportation, and 
the Alcohol and Gaming Commission of Ontario 
for conducting our fieldwork and testing user fees. 
In the 2008/09 fiscal year, these three ministries 
and one agency collected $1.7 billion in user fees— 
or 78% of all fee, licence, and permit revenues 
reported in that year’s Public Accounts. We sampled 
the more significant revenue-generating fees, with 
revenues totalling about $1.3 billion, charged by 
these ministries and this agency for review. In 
addition, we performed fieldwork at the Ministry 
of Finance, which provides operational support to 
the Treasury Board of Cabinet by reviewing all fee 
submissions from ministries and agencies and is 


RA 
a 





Total $2,176.2 


involved in recommending changes to policies and 
fee structures and amounts. 

Our audit fieldwork at these ministries and this 
agency assessed the total fees they charged for a 
related service. For instance, we selected all the 
fees that the Ministry of the Environment charged 
relating to hazardous waste, such as various char- 
ges for registering a hazardous-waste-producing 
site and for transporting hazardous waste. We also 
interviewed ministry and agency staff; examined 
records, documents, and policies in use; observed 
and tested operations; and reviewed relevant stud- 
ies, statistics, and major contracts. 

We also researched user fees in other jurisdic- 
tions. Because Quebec had recently issued a gov- 
ernment-task-force report on public user fees, we 
visited Quebec’s Ministry of Finance and discussed 
with senior management their recent develop- 
ment of a new user-fee framework to guide policy 
and operational decisions. At the federal level, we 
met with management at the Treasury Board of 
Canada Secretariat, who shared their perspectives 
on federal legislation, policy, and oversight on user 


fees charged by departments. We also met with the 
Office of the Auditor General of Canada regarding 
certain similar work it had done recently. We also 
reviewed Supreme Court of Canada decisions 
involving user fees from 1998 to the present. 

We also engaged on an advisory basis the 
services of an independent expert in public policy, 
who has recent significant experience pertaining to 
government user fees in the province of Quebec. 

In recent years, the three ministries’ internal 
auditors have conducted a number of audits rel- 
evant to our review of user fees; these audits have 
included tests and assessments of management’s 
compliance with required policies and procedures. 
These audits were helpful and of sufficient quality 
to allow us to reduce the extent of our work in cer- 
tain areas, such as whether internal controls over 
the collection of fees were adequate. 





A 1998 Supreme Court of Canada decision 
concluded that user fees could be considered 
unlawful and therefore may be repayable if they 
were determined by a court to be a tax that was 
not established by enacted legislation or if the fee 
amounts charged were excessive and did not have 
a reasonable relationship to the cost of the services 
provided. Although the Ontario government has 
taken some actions over the past decade to help 
address this ruling, there are still fee revenues from 
alcohol, gaming, and registration services of over 
$500 million annually that may be at risk because 
they may not fit the Supreme Court’s criteria for 
valid fees. 

The Non-Tax Revenue Directive (Directive) 
established in 1991 is intended to maximize the 
Ontario government’s non-tax revenues, includ- 
ing user fees, and ensure that ministries regularly 
review services and rates and keep non-tax revenue 
rates up to date. However, we found that the 
existing processes were, for the most part, not 
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effective in achieving the Directive’s goals. In addi- 
tion, unlike user-fee legislation in place federally 
and in some other provinces, Ontario’s existing 
policies and procedures lack transparency and 
public involvement in key decisions about changes 
to user-fee rates, nor is there sufficient public 
reporting on fees collected, their use, and the costs 
associated with providing the fee-related services. 

A key principle of the Directive is that when 
it is reasonable and practical to do so, the cost of 
providing services to the public should be borne by 
those who benefit from the service. The Ministry 
of Finance’s Costing and Pricing Policy, established 
in 2004, generally requires that the full cost of 
providing services—along with factors such as 
government priorities, the user’s ability to pay for 
the service, and other cost/benefit factors—be 
considered when establishing the user-fee rates. In 
2008, as part of the Budget process, the Ministry 
of Finance took the initiative of requiring that all 
ministries report on their fee revenues and their 
estimate of the costs of providing the fee-related 
services so it could evaluate opportunities for 
enhanced cost recovery. This one-time review 
indicated that most fees were not set at levels that 
would result in full cost recovery. Overall, fore- 
casted revenues did not recover about $522 million, 
meaning that less than 75% of the costs identified 
for these fee-related services was being recovered. 
In cases where ministries decide not to charge 
the full cost of a service—such as when it is not 
practical or economical to do so, or users cannot 
afford to pay—ministries are required to document 
the reasons for setting fees at reduced rates. For the 
most part, this was not being done. We also noted 
that, compared to most other provinces, Ontario 
collects less in terms of percentage of total revenues 
obtained from user fees and user fees charged ona 
per capita basis. 

In addition, there were generally no recurring 
processes in place to keep fee rates up to date, as 
is required under the Directive. We noted many 
examples of fees that have had no rate increase for 
10 to 20 years, despite the fact that the fees were set 
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at amounts that recovered only from 23% to 45% of 
the full costs of providing the services. In looking at 
other provinces, we noted that Nova Scotia adjusts 
user fees annually according to changes in the Con- 
sumer Price Index, and, starting in 2011, Quebec 
will systematically update its user fees using the 
same indexation rate it uses for personal income 
taxes. 

Some fee-related services are provided both in 
person and electronically—via the Internet or at 
electronic kiosks located at a number of publicly 
accessible locations, such as shopping malls, 
throughout the province. Ministry of Finance 
guidelines require ministries to discount fees for 
services provided electronically to encourage their 
increased use by the public. Services delivered elec- 
tronically can typically be delivered at a lower cost 
than over-the-counter services. However, we noted 
that no discounts were offered by the Ministry of 
Transportation for driver and vehicle registration. 
On the contrary, services at electronic kiosks incur 
a so-called “convenience” surcharge of one dollar 
per transaction for such services. 

Effective January 1, 2010, a new directive 
will apply to all provincial services regardless of 
whether a fee is charged or not. The directive will 
set out new common standards and will require 
that ministries establish program-specific standards 
for services offered, for monitoring and measuring 
the quality of service provided, and for communi- 
cating to users of services the actual level of service 
achieved. 

We concluded, based on our work and that of 
the internal audit services of the three ministries 
we visited, that internal financial controls over the 
fees collected by the three ministries were generally 
satisfactory. 


Detailed Audit Observations 





POLICY AND CONTROL FRAMEWORK 
OVER USER FEES 


Legislative and Administrative Context 


The legal foundation for setting and collecting taxes 
and fees is laid out in various sections of the Con- 
stitution Act, 1867. An important requirement that 
a government must have before it can impose a tax 
is parliamentary approval in the form of enacted 
legislation. Taxes may be imposed without any 
specific association to a particular good or service 
and can be for any amount. Unlike taxes, user fees 
are specific charges linked to the cost or value of 
particular goods or services that an individual 

or organization receives. User fees are typically 
imposed by a regulation enabled by an act or by 
Order in Council, which is a notice of an adminis- 
trative decision issued by the Lieutenant Governor 
but originating with Cabinet. 

In Ontario, the Treasury Board Act, 1991, gives 
legislative authority to the Treasury Board of Cab- 
inet to determine fees or charges by most ministries 
and certain agencies. In some cases, legislation 
gives a minister authority to set fees but Treasury 
Board still retains final approval. The Treasury 
Board is supported by the Ministry of Finance, 
which develops administrative policies regarding 
user fees and provides analysis and support for 
the annual Results-based Planning and Estimates 
process. This process requires that ministries and 
agencies report to the Ministry of Finance on their 
expenditure and revenue estimates, including any 
changes to their existing user fees or requests to 
establish new fees. 

In 1991, the Management Board Secretariat 
issued the Non-Tax Revenue Directive (Directive), 
which applies to user fees collected by all ministries 
and certain agencies. The Directive’s stated purpose 
is to: 


@ maximize the Ontario government’s non-tax 
revenue; 

@ ensure that ministries keep non-tax-revenue 
rates up to date; 

@ ensure that ministries review all services regu- 
larly and consider whether to establish new 
revenue rates or discontinue existing ones; 
and 

e enhance customer service. 

The Directive requires that those who benefit 
from a service should pay for the cost of providing 
that service when it is reasonable and practical 
to do so. A ministry is not required to establish a 
fee if it has determined that collecting revenues is 
impractical or uneconomical, where charges would 
severely undermine program objectives, where 
no specific user group can be identified, or where 
the users cannot afford to pay. When charges are 
deemed appropriate, the amount of the fee is to 
reflect program costs, program objectives, and 
government-wide priorities. Ministries are required 
to establish and record the criteria and calculations 
used to determine the amount of each fee or charge. 


User Fees versus Taxes 


A Supreme Court of Canada decision of more than 
a decade ago distinguished between fees and taxes 
and ruled that certain fees were invalid because 
they were actually an unauthorized tax. Specific- 
ally, in 1998, the Supreme Court of Canada decided 
that certain probate fees charged by the province 
of Ontario to the estate of Donald Eurig were in 
fact a tax on the estate assets and not a fee. In its 
decision, the Supreme Court laid out criteria to 
distinguish a fee from a tax. These included that, 
for a fee to be constitutionally valid, there must be 
a reasonable relationship between the cost of the 
service provided and the amount charged. It ruled 
that the probate fees in question had no relation- 
ship to the cost of the service provided, nor were 
they valid taxes because they did not originate in 
an act approved by the Legislature. The Supreme 
Court suspended the decision for six months to 
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enable the province to address the issue, which it 
did by implementing the Estate Administration Tax 
Act, 1998, replacing the probate fee with an estate 
tax. The Act legislated taxes retroactively to 1950 at 
rates that would produce the same revenue as the 
probate fees had generated. 

Since that time, although the Ontario govern- 
ment has taken some steps to address this, the 
actions taken have not been sufficient to make 
certain that all the fees it charges are legally fees 
and not, in fact, taxes. As a result, in our opinion, 
significant provincial revenues may still be at risk 
of being declared an invalid tax and at risk of being 
potentially repayable. . 

In response to the Eurig decision and to improve 
decision-making overall in the government, in 2004, 
the Ministry of Finance did develop a Costing and 
Pricing Policy, along with guidelines for its imple- 
mentation. The policy, which applies to all minis- 
tries and certain agencies, requires that the costing 
and pricing of services be in accordance with all 
relevant legislation. For costing, the policy requires 
that costs be determined and records maintained 
for all services, and it specifies the manner in which 
the costs are to be arrived at. For pricing, the policy 
requires that the full cost of delivering the service 
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be considered, along with other considerations, 





including, government priorities, clients’ ability to 
Pay, access to service, and whether a specific user 
group could be identified that derives a benefit 
from the service that the general population does 
not. The policy also requires that, where goods or 
services of the government are comparable to those 
of other jurisdictions or in competition with those 
of the private sector, decision-makers must receive 
benchmarking comparisons. As part of their annual 
Results-based Plans, ministries are to provide the 
Ministry of Finance with an explanation if a revenue 
source greater than $1 million changes by 20% or 
more. They must also include costing information in 
any submission to Management Board of Cabinet for 
approval of any new or proposed service. 

We were informed that the government made 
some changes to existing fees to be compliant with 
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the Eurig decision. In 2006, the government elimin- 
ated the gallonage fee on alcohol sales to licensed 
establishments, which collected approximately 

$46 million annually based on volume of pur- 
chases, with no direct link to any actual costs. In fall 
2006, the government amended the Highway Traffic 
Act to make it clear that the fee revenues collected 
to administer the driver’s licence and motor vehicle 
program, which amounted to $760 million in 2006, 
could be used to fund highway infrastructure and 
maintenance costs. 

In January 2007, the Supreme Court of Canada 
found certain New Brunswick alcohol-related fees 
unconstitutional, which resulted in the repayment 
of six years’ worth of fees, totalling about $1 mil- 
lion, to the owner of an establishment. We were 
advised that this prompted the Ontario government 
to take a further look at its non-tax revenues. In 
summer 2007, the Treasury Board directed min- 
istries to review all their fees. As a result of this 
review, it recommended that the ministries should 
come forward with options to address any issues 
identified through this review as part of the Results- 
based Planning process. 

As part of the 2008 Budget process, the Treasury 
Board directed the Ministry of Finance to undertake 
a review of all non-tax revenue sources to evaluate 
opportunities for enhanced cost recovery. Each 
ministry was required to report back to the Ministry 
of Finance by June 15, 2008, with information on 
opportunities to increase cost recovery for existing 
fees—in compliance with the Eurig decision and 
the Costing and Pricing Policy—and to identify 
opportunities for new fees that would be Eurig- 
compliant. Documentation we received from the 
Ministry of Finance confirmed that all ministries 
reported on their revenues. We were told that 
certain alcohol-related fees collected by the Alcohol 
and Gaming Commission of Ontario (Commission) 
were part of a separate review and were excluded 
from the 2008 review of non-tax revenue sources. 


Alcohol- and Gaming-related User Fees 
When planning our audit in October 2008, we 
noted that certain large alcohol and gaming fees 
collected by the Commission seemed to be signifi- 
cantly out of proportion to the related cost of their 
administration and therefore could be at risk of 
being non-compliant with the Supreme Court’s 
Eurig decision. In total, we identified over $470 mil- 
lion in fees the Commission charges annually that 
may be at risk of being declared non-Eurig compli- 
ant. This amounts to 21% of all provincial revenues 
collected from fees, licences, and permits. We 
reviewed fee revenues reported by other provinces 
and did not note similar large fee revenues that 
were alcohol- and gaming-related. 

In its March 2009 Budget announcement, the 
government said it was planning to introduce legis- 
lation to replace fees with taxes for various alcohol- 
related and other services, levies, and charges. 
However, it did not identify the specific fees. 


Other User Fees at Risk 
Our field visits to the Ministry of the Environment 
and Ministry of Transportation did not identify 
any large fees that we felt were at significant risk 
of being declared non-Eurig compliant. However, 
we noted that revenues collected by the Ministry 
of Government Services for certain registration 
services significantly exceeded the cost to provide 
the services by approximately $60 million, which is 
six times more than the costs to deliver them. At the 
time of our audit, the Ministry of Government Ser- 
vices had not established an action plan to address 
this issue. 


_ RECOMMENDATION 1 


To ensure that user fee revenues are not at risk 
of repayment because they are unconstitutional, 
the Ministry of Finance should obtain the legal 
assurances it needs or consider legislated or 
other changes that would protect the validity of 
these revenues. 


The Ministry of Finance acknowledges the 
Office of the Auditor General’s recommendation 
and will continue to review processes and take 
steps to protect the validity of these revenues. 
The 2009 Budget noted that the government 
plans to introduce legislation to replace various 
alcohol and other fees, levies, and charges with 





taxes to enhance their operational structure and 
legislative clarity. The government proposes to 
introduce this legislation at the earliest opportun- 
ity. The Ministry will continue to monitor non-tax 
revenue fees, including through the annual 
Results-based Planning and in-year reporting 
processes. As part of the 2010/11 Results-based 
Planning and future years reporting processes, 
ministries must report annually on non-tax 
revenue collected in the previous fiscal year, 
expected revenues for the current fiscal year, 

and revenue projections reflecting the multi-year 
planning period. Any new in-year submissions/ 
proposals on non-tax revenue must be compliant 
with applicable case law. 


Policy Framework and Processes 


A best practice with respect to user fees is for 
governments to set, preferably through legislation, 
an overarching policy framework that provides 
transparency and clarity, and promotes consistency. 
Such a framework would lay out criteria for impos- 
ing new fees and modifying existing ones; establish 
how costs, prices, financial targets, and service 
standards are to be determined; and clarify expect- 
ations for financial performance, service standards, 
and reporting. When government policy require- 
ments are expressed in an act of the Legislature—in 
other words, as a law—this sends a clear message, 
not only to ministries and agencies but also to the 
public, on where a government stands with respect 
to charging fees for goods and services provided. 
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In Ontario, there is no overarching legislation 
regarding user fees. As previously stated, the admin- 
istration of fees is governed by the Non-Tax Revenue 
Directive, established in 1991, the Costing and 
Pricing Policy, established in 2004, and the annual 
Results-based Planning process. However, we found 
that the existing processes could not be relied upon 
to maximize and keep non-tax revenues up to date, 
and it was too soon to tell if the changes to the 
Results-based Planning process would remedy this. 
Beginning in the 2008/09 fiscal year, the Results- 


based Planning process was updated to require 


that ministries identify revenues that were not in 
compliance with the Eurig decision and to require 
that any decisions on user fees consider the full cost 
of services instead of being based on the previously 
used direct-program-only costs (this is more fully 
described in the Enforcement and Compliance 
Costs section). We noted that user-fee policies and 
processes were often largely driven by the pressures 
and timelines associated with the provincial Budget 
or by reaction to Supreme Court decisions. 

The Results-based Planning process requires 
that ministries report to the Ministry of Finance any 
changes to fee rates, new fees, and cases where the 
revenues generated by a fee-related service exceed 
$1 million and will change year-over-year by 20% 
or more. The responsibility for regular reviews 
and consideration of whether to establish new 
revenue rates or discontinue existing ones belongs 
to individual ministries. Thus, under Results-based 
Planning, there is no reason for fees with stable 
revenues to be reviewed; therefore, the appropri- 
ateness of their rates might not be reconsidered for 
a very long time. 

Even if the Non-Tax Revenue Directive was 
achieving its goals, it might not produce the desired 
results overall because more than half of all user-fee 
revenues come from fees that appear to be exempt 
from the Directive. According to the Directive, 
non-tax revenue fees established and revised by the 
Ministry of Finance and announced in the Budget 
do not have to comply with the Directive. Of the 
$2.2 billion in user fees collected in 2007/08, we 


Chapter 3 * VFM Section 3.05 





150 J 2009 Annual Report of the Office of the Auditor General of Ontario 





Chapter 3 ¢ VFM Section 3.05 


identified about $1.3 billion of that revenue from 
two ministries we visited for which the Directive 
technically does not apply. These fees, considered 
to be under Ministry of Finance control, include 
approximately $848 million of driver’s licence and 
vehicle registration fees and $455 million of brew- 
ers’ fees. In our discussions at the Ministry of Trans- 
portation and the Commission where these fees are 
collected, senior management told us that decisions 
over changes to these fees were typically made by 
the Ministry of Finance and often only came to the 
ministries’ and Commission’s knowledge when 
Budget announcements were made public due to 
the confidentiality of the Budget process. 

In addition, we identified several other concerns 
with the existing policies and procedures over user 
fees, including: 

@ There is little or no public involvement in 
decisions relating to existing user fees. With 
respect to new fees, ministries must consult 
stakeholders, but there are no required mech- 
anisms for public input. 

e@ The processes used to review and modify 
user fees periodically are not transparent to 
the public; the public is typically made aware 
of changes to user fees only when they are 
announced in the provincial Budget. 

@ Cost recovery targets for specific user fees 
have generally not been established by the 
Treasury Board, the Ministry of Finance, or at 
the ministry level, to guide future decisions, 
such as the extent to which clients benefiting 
from a fee-related service should be required 
to cover its costs, and whether fee rates should 
be regularly updated for inflation and cost 
fluctuations. 

e There is no periodic or annual public 
reporting on fees, other than aggregate 
amounts included in the Public Accounts. For 
the Public Accounts, ministries must report 
on their significant fee, licence, and permit 
revenue, but they do not have to relate that 
information to the costs incurred. More com- 
prehensive reporting on fees collected, their 


use, and the costs of their associated services 
would help to demonstrate transparency in 
this area. 

In contrast to Ontario, several Canadian jurisdic- 
tions have recently enacted or announced their 
intention to enact legislation providing clarity, 
transparency, and consistency over how user fees are 
managed and over public consultation and reporting 
on services provided, costs incurred, and revenues 
raised. Specifically, we noted the following: 

e@ The federal government enacted the User Fees 
Act in 2004 to strengthen accountability, over- 
sight, and transparency in the management 
of user fees. The legislation defines a user fee, 
incorporating the notion that a direct benefit 
or advantage is conferred to the person paying 
the fee. It sets out requirements for depart- 
mental implementation of new or amended 
user fees. Before implementing a proposed 
user fee or changing an existing fee, the 
government service provider must explain to 
clients the reasons for the fee and the cost and 
revenue elements involved. All clients must 
be given a reasonable opportunity to provide 
input and, if necessary, an independent 
advisory panel can be established to address 
the issues raised. In addition, the legislation 
requires that service standards be established 
and actual performance relating to amended 
or new fees be reported to Parliament annu- 
ally. Reporting on fees must include the 
full costs incurred, revenue received, date 
of the last fee increase, and information on 
stakeholder consultations. The legislation also 
requires an explanation when a fee amount 
proposed is higher than that found in another 
jurisdiction with which comparisons of fees 
are made. In addition, to complement federal 
legislation, the Treasury Board of Cabinet Sec- 
retariat provides policies and guidance on the 
processes for proposing user fees, setting fees 
and service standards, and reporting on new, 
amended, and existing fees. This information 
is available for public review on its website. 
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e@ Nova Scotia enacted in 2007 the Fees Act, 


; andc i i ici ini 
which stipulates that no fee increase is ompliance with policies, the Ministry of 


Finance should research legislation, policies, 
and processes in use or planned in other juris- 
dictions to identify best practices that could be 
applied in Ontario. It should also consider mak- 
ing available to the Legislature and the public, 
as some other provinces do, information on 
decisions related to user fees, such as the extent 
to which fees are expected to recover costs, and 


authorized unless the minister responsible 
notifies the Legislature and provides such 
details as the purpose of the fee, total revenue 
expected, and whether the fee rate is intended 
to recover full or partial costs. In recent years, 
the province has increased fees on a govern- 
ment-wide basis on April 1 at a rate tied to 
the Consumer Price Index. Fee increases have 


; requirements for proposing new fees and fee 
been reported in a public document. q ee ae 


increases. 
e New Brunswick enacted new user-fee legisla- 


roicningeeschaged yen 
governing fees charged by government 


departments and to address stakeholder con- The Ministry of Finance supports ongoing 

cerns that had been expressed about sudden efforts across government towards account- 

fee increases. In January 2009, the province ability, openness, and transparency in decision- 

published its first annual report on its fees, making. As part of these efforts, the Ministry 

which included for each fee the legislative will review practices in other jurisdictions by . 

authority, current amount, effective date and 2010/11 and will consider their applicability io 

amount of any increases, expected annual rev- in view of Ontario’s current public policies and | 5 

enue, and any changes in expected revenue. considerations, such as government priorities, 3 

The report also explains what any new fees economic and social factors, tax-base, and other = 

or changes in fee amounts are intended to cost/benefit factors. Any new policies must be = 

accomplish. approved by the Treasury Board/Management o 
@ Inits 2009/10 Budget, the Quebec govern- Board of Cabinet. We will continue to review | S 

ment announced its commitment to imple- each fee on its own merit. 5 





ment overarching user-fee legislation. The 
stated purpose of the legislation is to enhance 


ey i 
the funding of services to maintain qua ee | FEE PRICING AND COSTS 
and ensure transparency and accountability in 

the fee-setting process. By 2012, the govern- Cost Recovery for Services 


ment will systematically evaluate the costs of ny ey 
4 . Our audit indicated that, for the most part, at the 


services for which existing or potential user a ve 4 
three ministries we visited, most fees were not 


fees apply, determine self-financing targets 


Bh ; ; set at levels that would result in full cost recovery 
for each fee-based service, index increases in 


for the related services provided and there was 
fees annually at the same rate as any increase Moen I 

: no documentation available at the ministries, as 
to the personal taxation system, and report 


; . required by the Directive, to indicate the rationale 
annually to the public for accountability. 


for charging less than full cost. Because the costing 
of the fee-related services being provided was not 
RECOMMENDATION 2 ; ave 
being calculated consistently and periodically, it 
To improve accountability, openness, and was not possible to determine with accuracy the 
transparency in decisions related to user fees extent that overall fees recover their costs at these 
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three ministries. However, as an indication and one 
estimate for all ministries, the 2008 Budget review 
of costs and revenues, conducted to evaluate oppor- 
tunities for enhanced cost recovery, showed about 
$522 million less revenues than fee-related costs. 
Overall, total forecasted recoveries in all ministries 
were less than 75% of the costs being incurred to 
deliver the fee-related services. 

As previously mentioned, both the Non-Tax 
Revenue Directive and the Costing and Pricing Policy 
require that consideration be given to setting fees 
to recover the full cost of the fee-related service so 
that those who benefit from a service would pay 
the cost of providing it where it was reasonable and 
practical to do so. If a ministry decides not to charge 
the full cost for services—if, for instance, it would be 
uneconomical to do so, or if users cannot afford to 
pay—it must document its rationale. 

Many fees have been in place for many years, 
and we found that there was generally no docu- 
mentation relating to the setting of fees at rates that 
would cover the costs of the related service. In other 
words, the ministries were not periodically formally 
reviewing fees to ensure that they fully recovered 
the associated costs or, if they had decided not to 
attain full cost recovery, they were not documenting 
the reasons for their decision. It should be noted 
that the 2008 Budget process’s one-time review of 
all non-tax revenue sources to evaluate opportun- 
ities for enhanced cost recovery helped to address 
this concern, in that all ministries were required to 
provide an assessment of what changes were needed 
to fee rates to achieve 100% cost recovery and what 
the impact would be on stakeholders. 

We found that several recently established fees, 
on the other hand, were better supported in that 
key decisions—such as the basis for establishing 
the fee rate, the costs associated with the fee, and 
whether the fee is Eurig-compliant—were well 
documented. These new fees, however, represent 
less than 0.1% of total user fee revenues. 

At the Ministry of the Environment, we noted that 
cost-recovery targets were set for certain programs. 
When “Drive Clean”—a mandatory vehicle-emis- 


sions inspection and maintenance program—was 
established in 1999, the Ministry set a target of 100% 
recovery of the cost of the program. In 2007/08, 

this amounted to $31 million, which we noted the 
program was generally achieving. The Hazardous 
Waste Cost Recovery Fee program was introduced on 
January 1, 2002, to recover fully the costs of manag- 
ing hazardous waste in the province and to encour- 
age generators to reduce the amount produced. The 
Ministry committed to reviewing the program within 
three to five years, but it had not yet completed this 
at the time of our audit. We compared the costs of 
this program identified for us by the Ministry against 
revenues received and found that only about $6 mil- 
lion of the estimated $19 million in costs—about 
31%—was recovered in 2007/08. The $13-million 
shortfall was being covered by the province’s general 
revenues. We had previously raised this issue in our 
2007 audit of this program. 

As part of our audit, we compared Ontario’s fee 
revenues with those of the federal government and 
the other provinces. Figure 2 shows that, compared 
to Canada and the other provinces, Ontario ranks 
as second-lowest (tied with Saskatchewan) in the 
percentage of its total revenues that come from 


Figure 2: User Fees as Percentage of Total Revenue— 
Ontario vs. Other Jurisdictions, 2007/08 


Source of data: Public Accounts of Canada and all provinces 


35 
3.0 
2.5 
2.0 
LBS 
1.0 
0.5 


0.0 





Note: Because of variations in how fees are classified by various jurisdictions, 
certain fees were omitted that were not applicable to Ontario. Also, we 
deducted Ontario’s alcohol and gaming-related fees because other provinces 
did not report these types of revenues in a similar manner. 


user fees. Figure 3 illustrates that Ontario residents 
pay the least amount per capita in user fees for 
government services. These statistics may indicate 
that the Directive may not be achieving its intended 
objective of maximizing non-tax revenues by having 
users who benefit from fee-related services pay the 
full cost of the services where its is reasonable and 
practical to do so. 

a AE lee igs SE ee ce 2 
Figure 3: User Fees Per Capita—Ontario vs. Other 
Jurisdictions, 2007/08 

Source of data: Statistics Canada and other provinces’ Public Accounts 
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Note: Because of variations in how fees are classified by various jurisdictions, 
certain fees were omitted that were not applicable to Ontario. Also, we 
deducted Ontario’s alcohol- and gaming-related fees because other provinces 
did not report these types of revenues in a similar manner. 


RECOMMENDATION 3 


To meet the intent of the Non-Tax Revenue 
Directive that non-tax revenues be maximized, 
user-fee rates should be set at levels that would 
recover the costs of providing services where 
it is reasonable and practical to do so. Where 
full costs are not being recovered, there should 
be adequate documented rationale. As well, 
the Ministry of Finance, in conjunction with 
the other ministries and with Treasury Board 
approval, should consider establishing target 
cost-recovery ratios for services for which full 
costs are not being recovered. 
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The decision on the appropriate level of costs 
that should be recovered is based on govern- 
ment policy, under the purview of the Treasury 
Board. This is done on a case by case basis 

and with due consideration of other factors, 
including socio-economic and public policy 
administration, to balance the overall benefits 
to the public interest. Current policies allow 
setting fees at various levels, including below 
full cost-recovery. This is consistent with public 
policy choices to support provision of certain 
public goods, or to send price signals that 
impact consumer choice (for example, as an 
incentive or a deterrent). The Ministry will give 
due consideration to the need for setting target 
cost-recovery ratios for any fees that are set 
below full cost-recovery and will remind min- 
istries of the requirements to retain documents 
relating to the setting of fees to ensure that they 
are readily available for future reviews. 


Updating Fee Amounts 


The Directive requires that ministries keep non- 
tax revenue rates up to date, review all services 
regularly, and consider whether to establish new 
revenue rates. We noted that, except for a few 
cases, the three ministries did not have any regular 
processes in place to update fees, such as processes 
to update fees for changes in costs or inflation. As 
previously mentioned, at the request of the Ministry 
of Finance, ministries conducted a comprehensive 
analysis of fee revenues and associated costs as part 
of the 2008 Budget process in an effort to identify 
opportunities to enhance revenues. However, we 
were informed that this was the first time such an 
assessment was conducted and that no process is in 
place for a regular yearly assessment of all user fee 
revenues in relationship to service costs. 

A regular process for updating fee rates would 
meet the requirement of the Directive and help to 
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ensure that rates are maintained at levels closely 
matching intended recovery rates of actual costs. 
This would also help avoid the need to impose large 
rate changes to address cost increases that had 
occurred over a number of years. Similarly, for costs 
that have decreased—because of, for example, 
advancements in technology—the savings could 
also be passed along to the users of the fee-related 
services. 

We identified a large number of fees for which 
no rate increases or inflationary adjustments had 
been made for long periods of time, even though 





the fee revenues did not cover the associated costs. 
Figure 4 provides a sample of fees we identified 
during our visits to the three ministries and the 
Commission that have not been adjusted for many 
years—in some cases, two decades—and were 
recovering less than half of their related costs. 

We noted that Nova Scotia adjusts user fees 
annually to account for changes in the Consumer 
Price Index, and, by 2012, Quebec plans to system- 
atically update its user fee charges at the same rate 
that it indexes personal income taxes. 


Figure 4: Examples of Fee-related Services without Recent Fee Rate Adjustments 
Source of data: Ministries of Finance, the Environment, and Transportation, and the Alcohol and Gaming Commission of Ontario 












Ministry or Agency Service 






















Annual 
Revenue Rate of 
(S$ million) Recovery (%) 


Last Annual Cost 
Update ($ million) 





Alcohol and Gaming _—_ gaming registration-casino employee 1999 4.0 1.8 45 
Commission of gaming registration-casino suppliers 1992 8.3 2.4 29 
aneue liquor licence renewals-2 & 3 year 1997 1 3.8 29 
Environment certificate of approval 1998 17.9 704 39 
Government Services name change—foreign-based corporation 1998 0.3 0.1 33 

name change—individual 1990 2.4 ilUa 46 
Transportation commercial international registration 1988 1577 479 30 

plan-Ontario carriers 

commercial international registration plan 1988 96.7 9.3 53 


-non-Ontario carriers 





RECOMMENDATION 4 





To help ensure that ministries comply with 
existing policies requiring them to keep fee 
rates up to date with costs being incurred, the 
Ministry of Finance should work with ministries 
to establish regular processes for identifying 
changes in the costs of service delivery and for 
making formal recommendations to the Treas- 
ury Board for regularly updating fee rates. 





The Ministry of Finance currently has processes 


in place to identify the cost of service delivery 
as part of its annual planning and in-year 


processes. Periodic reviews of fees have been 
undertaken to determine which fees may 
require adjustment. The Ministry will work 

with ministries to consider appropriate busi- 
ness cases, as part of the annual Results-based 
Planning and in-year processes, for fee adjust- 
ments where the costs of service delivery have 
changed. All such recommendations must be 
approved by the Treasury Board/Management 
Board of Cabinet. Because an increase in costs 

is only one of the factors considered in setting a 
fee, it will not automatically result in an increase 
to the fee. Government priorities, socio-eco- 
nomic factors, the user’s ability to pay, and other 
cost/benefit factors are also considered when 
establishing user fee rates. 





Fees for Electronic Service Delivery 


The annual Results-based Planning Technical 
Guide issued by the Ministry of Finance states that 
increased use of electronic service channels should 
be encouraged and that, where services are offered 
both electronically and over the counter, fees for 
electronic service are required to be discounted. It 
is typically less expensive to deliver services elec- 
tronically than over the counter. We noted that the 
Ministry of Government Services discounts certain 
services, such as business registrations, if they are 
delivered online. 

The Ministry of Transportation offers both in- 
person counter service and, using ServiceOntario 
of the Ministry of Government Services, electronic 
Internet-based service for vehicle validation tag 
renewals, driver and vehicle records, and personal- 
ized and graphic plates. However, it does not offer 
a discount on the electronic services. Moreover, it 
provides these same services at ServiceOntario’s 
electronic kiosks located at a number of publicly 
accessible sites, such as shopping malls, through- 
out the province—but charges an extra fee. A 
“convenience” surcharge of one dollar is added by 
ServiceOntario to the cost of each transaction at the 
electronic kiosks. Convenience surcharge revenues 
totalled about $842,000 in 2008/09. We asked the 
Ministry how it justifies this surcharge and why its 
Internet-based service is not discounted, but it was 
not able to provide any information comparing its 
kiosk and Internet costs with those of its over-the- 
counter service. 


RECOMMENDATION 5 


The Ministry of Transportation, in conjunction 
with the Ministry of Government Services, 
should compare its costs for delivering services 
via electronic kiosk and online with those of 
over-the-counter, in-person service delivery to 
establish whether “convenience” fees added 

to electronic kiosk services are justified and 
whether kiosk and online service delivery 
should be discounted. 
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The Ministry of Transportation and the Ministry 
of Government Services acknowledge the Aud- 
itor General’s recommendation. The Channel 
Pricing Strategy requires ministries to ensure 
that services offered both electronically and 
over-the-counter be cost effective. However, 
some services, such as kiosks, are considered a 
premium service, for which a fee is charged to 
acknowledge the cost of this service. ServiceOn- 
tario and the Ministry of Transportation work in 
partnership to offer in-person counter services, 
kiosks, and electronic Internet-based services to 
the public. Both the ownership and the manage- 
ment of channel service delivery, including con- 
tract management, were transferred from the 
Ministry of Transportation to ServiceOntario in 
2007. The Ministry of Transportation is working 
closely with ServiceOntario as they progress 
with their business strategy for moving consum- 
ers to the electronic channel. This will include 
the review of the convenience fee in the context 
of contractual obligations with private-sector 
providers, existing regulations and legislation, 
Ministry of Finance policies, and costing across 
all channels. ServiceOntario will also consider 
the full impact of all government costs, includ- 
ing information technology systems and contact 
centre support, to ensure appropriate rates 

for electronic services as part of the 2010/11 
Results-based Planning process. 


Enforcement and Compliance Costs 


The Costing and Pricing Policy’s guidelines are used 
by ministries to determine the full cost of their fee- 
related services to ensure that fee-related decisions 
are based on accurate, complete, and consistent 
costing information. Costs that are to be allocated 
include direct program costs, direct program 
support costs, ministry corporate costs, indirect 
costs incurred by other ministries, and, where 
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appropriate, risk costs (for example, ministry settle- 
ment payments for common lawsuits). Although 
ministry financial systems typically record direct 
program costs accurately, ministry staff usually 
need to identify and make specific decisions about 
the appropriate allocation of other costs to the fee- 
related service. 

The guidelines give no instruction on how min- 
istries should allocate enforcement and compliance 
costs to fee-related services for costing purposes. 
For many programs, enforcement and compliance 
costs are a significant and integral part of the ser- 
vice being delivered. For instance, when a driver’s 
licence is issued under the Highway Traffic Act, 
allowing individuals to drive vehicles on a highway, 
the fees associated with this service should also 
cover the costs of ensuring that drivers comply with 
the Highway Traffic Act, which would include the 
costs of the OPP enforcing this Act. 

We found that enforcement and compliance 
costs were not consistently applied to the fee- 
related services. For example, in its assessment of 
its non-tax revenues prepared as part of the 2008 
Budget process, the Ministry of the Environment 
included for the Drive Clean program almost 
$1 million of investigation, enforcement, and com- 
pliance costs from its Investigation and Enforce- 
ment Branch and Sector Compliance Branch. 
However, the Ministry did not include any compli- 
ance or enforcement costs in its costing assessment 
related to the fee that waste generators pay to 
obtain Certificates of Approval. The Ministry’s 
assessment provided to the Ministry of Finance 
indicated that its costs were almost $18 million; 
the total would have been about $5 million more if 
enforcement costs had been included. 

Similarly, the Ministry of Transportation allo- 
cated $27.5 million in compliance and enforcement 
costs it directly incurred to arrive at its full costs for 
commercial carrier fee-related services. These com- 
pliance and enforcement expenditures included the 
cost of the Ministry’s Transportation Enforcement 
Officers who carry out inspections of commercial 


motor vehicles at its truck inspection stations, by 
patrolling, and at carriers’ facilities. However, the 
Ministry did not include as part of its reported 
$760 million costs for the driver and vehicle licens- 
ing and registration programs any costs from the 
Ontario Provincial Police, which spends approxi- 
mately $189 million a year patrolling and enforcing 
laws on Ontario highways. Although the OPP 
operate under a different ministry—the Ministry of 
Community Safety and Correctional Services—the 
Costing and Pricing guidelines specifically allow 
for indirect costs incurred by other ministries to be 
included when determining the full costs of fee- 
related services. 


| RECOMMENDATION 6 





To ensure that accurate and consistent informa- 
tion is available for making informed decisions 
on fee rates, the Ministry of Finance should 
amend its Costing and Pricing Policy and guide- 
lines used by ministries to require that compli- 
ance and enforcement costs be appropriately 
considered when determining the full cost of 
fee-related services. 





The Ministry of Finance agrees to review the 


Costing and Pricing Policy and guidelines by 
2011/12 with respect to considering enforce- 
ment and compliance costs, where applicable, 
when determining the full cost of fee-related 
services to ensure more consistent application of 
the policy. The Ministry of Transportation and 
the Ministry of the Environment will continue 

to work closely with the Ministry of Finance 

to ensure that any changes to the policy and 
guidelines, or clarification of costs relating to 
compliance and enforcement, will be considered 
and allocated appropriately when determining 
the full cost of fee-related services. 


REVENUE COLLECTION 


On the basis of our testing of a sample of fees, we 
concluded that internal financial controls over rev- 
enue collection established by the three ministries 
were generally satisfactory. We reviewed internal 
audit reporting, where available, and found that, 
where deficiencies were noted, timely corrective 
action was taken. In addition, we informed the min- 
istries of several less significant audit observations 
and made recommendations for improving internal 
controls over fee collection and accounting. 

The Ministry of the Environment’s Drive Clean 
program is administered by a private company that 
is responsible for collecting revenue from author- 
ized emission-testing facilities and forwarding 
these revenues to the Consolidated Revenue Fund. 
Drive Clean program revenues were about $32 mil- 
lion in 2008/09. We noted that the last time the 
Ministry hired independent auditors to assess finan- 
cial controls over Drive Clean revenues collected 
by the private company was in 2002, at which time 
controls were determined to be adequate. The 
Ministry informed us that it has undertaken no 
further assessments in the subsequent seven years 
because no significant changes have occurred to the 
program’s operating and financial procedures. 

In addition, the Ministry did not carry out any 
assessments to ensure that the revenues were 
reasonable, for example by predicting the revenue 
using the Ministry of Transportation vehicle regis- 
tration database. Given the significant amount of 
government Drive Clean revenue being collected by 
a private company, it would be prudent for the Min- 
istry to periodically obtain independent assurance, 
such as from an audit, that the appropriate amounts 
are being remitted and that internal controls 
established by the private company are adequate. 
The most cost-effective solution might be for the 
Ministry’s senior management audit committee to 
request that an audit be included in the 2010 work 
plan of the Ministry’s internal audit services. 
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RECOMMENDATION 7 


The Ministry of the Environment should obtain 
periodic internal or external audit and other 
assurances that the revenues collected and 
remitted by the private-sector operators of its 





Drive Clean program are accurate. 


The Ministry of the Environment appreciates 
the Auditor General’s recommendation and will 
include in its 2009/10 Audit Plan an internal 
audit of the revenues collected and remitted by 
the private-sector operators of the Drive Clean 
Program. In addition, the Ministry will set up 

a regular cycle of audits conducted by parties 
external to the program, such as the Ministry’s 
Internal Audit Branch, or an external contractor, 
to strengthen oversight of revenue collection 
and remittance. This plan will augment the cur- 
rent financial oversight the Ministry administers 
on a daily and monthly basis to ensure that 
appropriate financial controls are in place on 
the revenue collection and remittance of Drive 
Clean fees. 


Chapter 3 * VFM Section 3.05 





SERVICE STANDARDS AND REPORTING 


A new service directive intended to provide an 
updated, customer-focused framework is to apply to 
all ministries of the Ontario government effective 
January 1, 2010. The directive will set out new 
common standards and will require that ministries 
establish program-specific standards for services 
offered, for monitoring and measuring the quality 
of service provided, and for communicating to 
customers the actual level of service achieved. The 
directive applies to all provincial services, regard- 
less of whether a fee is charged or not. We noted 
that the federal user-fee legislation for new or 
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amended fees and Treasury Board of Cabinet Sec- 
retariat policies for existing fees impose additional 
requirements for federal departments, agencies, 
boards, and commissions that can impose fees for 
their services. For instance, standards are required 
that are comparable to those established by other 
countries with which a comparison is relevant and 
against which the performance can be measured. In 
addition, explanations must be provided to clients 
on how the user fee is determined and on its related 
costs and revenues. In this way, clients can clearly 
see the cost of the services they pay for in relation 
to what they receive. 

In Ontario, a ministry may choose to offer a ser- 
vice guarantee, providing compensation to a client 
if the promise or pledge of service is not met. We 
noted, for example, guarantees for certain registra- 
tion services for births, deaths, and marriages from 
the Ministry of Government Services. Federally, 
user-fee legislation mandates that, regardless of 
whether a specific service guarantee is offered, if in 
a particular fiscal year, the performance of a service 
for which a user fee is charged fails to meet the 
established service standards by more than 10%, 
in the following year, the user fee is to be reduced 
for all its clients, by a percentage equivalent to the 
unachieved performance, to a maximum of 50%. 


RECOMMENDATION 8 


To enhance accountability and reporting over 
ministries’ fee-related services, the Ministry of 
Finance, in conjunction with ministries, should 
identify and implement the best practices in use 
in other jurisdictions relating to establishing and 
publicly reporting service standards and actual 
service levels achieved. 





As the lead ministry in the development of the 
new Ontario Public Service Directive that comes 


into effect in January 2010, the Ministry of Gov- 
ernment Services is now supporting ministries 
in the implementation of the Directive’s require- 
ments. These include establishing program- 
specific service standards in consultation with 
clients, communicating the standards to clients, 
and subsequently measuring and reporting 
back on the achievement of those standards. 

As part of the service improvement program, 
the Ministry of Government Services, on behalf 
of the Ontario Public Service, also works with 
interjurisdictional organizations to conduct 
benchmarking studies on service quality, which 
enables the Ontario Public Service to compare 
its service quality to that in other jurisdictions. 


Chapter 3 
Section 






Homes 


Background 





Long-term-care homes in Ontario provide care, ser- 
vices, and accommodations to individuals unable 

to live independently and requiring the availability 
of 24-hour nursing care and supervision in a secure 
setting. There are more than 600 long-term-care 
homes in Ontario caring for about 75,000 residents, 
most of whom are over 65 years old. The long-term- 
care homes essentially become “home” for most of 
their residents. All homes fall within one of four 
categories: for-profit and not-for-profit nursing 
homes, charitable homes, and municipal homes for 
the aged, as illustrated in Figure 1. 

There is a high risk of infectious diseases spread- 
ing among residents of long-term-care homes 
(hereafter referred to as “residents” and “homes”) 
because they often share rooms with other resi- 

o Site ee ee 
Figure 1: Ontario’s Long-term-care Homes by Type, 
November 2008 


Source of data: Ministry of Health and Long-Term Care 


Home type # of Homes # of Beds 
nursing home (for profit) 353 fee 200 
nursing home (not-for-profit) 95 11,200 
charitable (not-for-profit) 54 | 7,500 
municipal (not-for-profit) 103 16,400 
Total 605 


<Als] Infection Prevention and 
Control at Long-term-care 


dents, and generally eat and participate in activities 
together. As well, residents generally have a higher 
risk than the population as a whole of acquiring an 
infection because they are older and more vulner- 
able to illness. Further, residents who are cogni- 
tively impaired may not always mention to staff 
their symptoms when they first appear, and may 
wander, both of which increase the opportunities 
for infectious diseases to spread. 

When a resident acquires an infection while in 
a home, it is considered a health-care-associated 
infection (HAJ), also called a “nosocomial infec- 
tion.” HAIs have a significant impact on both 
residents and the province’s health-care system. 
For residents, the impact of such infections can 
range in severity from not feeling well for a few 
days to requiring antibiotics or even being admitted 
to hospital. In severe cases, HAIs can cause death. 
Although there is no information available on the 
total number of HAIs that occur in Ontario’s homes 
each year, studies indicate that infection is one of 
the most common reasons for the hospitalization 
of residents. In fact, one U.S. study indicated that 
infection was the main medical reason for about 
27% of all hospital admissions of residents. 

Some HAIs are infectious diseases that can spread 
throughout a home. Figure 2 provides some back- 
ground information on four serious HAIs: Clostrid- 
ium difficile (C. difficile), febrile respiratory illness 
(FRI), methicillin-resistant Staphylococcus aureus 
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Figure 2: Four Infectious Organisms/ Diseases Acquired in Long-term-care Homes 
Prepared by the Office of the Auditor General of Ontario 


Clostridium 
difficile 
(C. difficile) 
bacteria 


Febrile 
Respiratory 
IIness (FRI) 
(e.g., colds, 
influenza, 
pneumonia) 


Methicillin- 
resistant 
Staphylococcus 
aureus (MRSA) 


Vancomycin- 
resistant 
enterococci 
(VRE) 


How Resident 


Initially Infected 


¢ resident takes 
antibiotics that 
reduce the 
normal levels of 
good bacteria in 
intestines and colon 


° this allows 
C. difficile bacteria 
to grow and 
produce toxins 


* person coughs or 
sneezes droplets 
containing disease- 
Causing organisms, 
which contact 
resident’s mouth, 
nose, or eyes 


° resident touches 
droplets and then 
touches mouth, 
nose, or eyes 


* Staphylococcus 
aureus (S. aureus) 
bacteria living on 
the skin, nose, 
or in the lower 
intestine cause an 
infection and resist 
a common class of 
antibiotics (many 
people who carry 
the bacteria do not 
have symptoms) 


enterococci 
bacteria in lower 
intestine and/or 
other areas (e.g., 
urine, skin) cause 
an infection and 
resist Vancomycin 
antibiotic (many 
people who carry 
the bacteria do not 
have symptoms) 


Examples of 
Possible Effects 


° diarrhea 


© more serious 
intestinal 
conditions (é.¢., 
colitis) that may 
require surgery 


° death in extreme 
cases 


* contact? 


° fever greater than 
Biehl & 

° new or worsening 
cough 

° shortness of breath 


° death in extreme 
cases 


° “droplet”? 
° contact* 


¢ skin infections * contact? 
that can quickly 
turn into deep 
abscesses that 
require surgical 
draining 

* infections in bones, 
joints, surgical 
wounds, the 
bloodstream, heart 
valves, and the 
lungs 

° death in extreme 
cases 


* urinary tract * contact’ 
infection or skin 


infection 


° death in extreme 
cases 


Transmission 


Possible 
Treatments 


* mild cases: 
may not 
require 
treatment 


* severe Cases: 


antibiotics 


© antibiotics 
when 
applicable 


* immunization 
prior to 
exposure 
to certain 
FRIs (as a 
preventative 
measure) 


° mild cases: 
may not 
require 
treatment 

° severe 
cases: other 
antibiotics 


¢ other 
antibiotics 


Other Concerns 





* can lead to outbreaks 
because many people in 
long-term-care homes take 
antibiotics 


* C. difficile spores are 
difficult to destroy because 
they are resistant to a 
number of chemicals 


* alcohol-based hand 
cleansers may not be as 
effective as soap and 
water 


* disease-causing organisms 
in droplets can live on 
surfaces for hours but 
are easy to kill with 
disinfectants and good 
hand hygiene 


* although infections caused 
by MRSA are not more 
serious than infections 
caused by S. aureus 
bacteria, there are fewer 
antibiotics available 
to treat MRSA-caused 
infections 


* bacteria can live on 
Surfaces for months 


* bacteria can live on 
surfaces for 5 days to 
weeks and on hands for 
several hours 


* bacteria are relatively easy 
to kill with disinfectants 
(provided the bacteria 
are in contact with the 
disinfectant for a long 
enough period) and good 
hand hygiene 


1. Contact can be from person-to-person touching and touching of contaminated surfaces on which there are spores, droplets, or bacteria. A person who 
acquires the infection through contact will not necessarily become ill (e.g., a person may become infected with C. difficile bacteria from a resident but have 
enough good bacteria to fight the C. difficile bacteria). 


2. “Droplet” transmission involves the infected person coughing or sneezing and causing droplets to come into direct contact with another person. 
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(MRSA), and vancomycin-resistant enterococci 
(VRE). Each of them can be transmitted through 
contact—that is, by touching an infected person ora 
surface on which the bacteria live. Therefore, hand- 
washing and cleaning and disinfecting surfaces that 
residents and staff come into contact with are critical 
to preventing the spread of these infections. The inci- 
dence of MRSA has approximately doubled and that 
of VRE more than tripled between 1999 and 2006, 
according to data reported by the Canadian Noso- 
comial Infection Surveillance Program. Although 
most of these infections were acquired in hospitals 
(insofar as the point of acquisition was known), 

8% of cases of MRSA and 3% of cases of VRE were 
acquired in long-term-care homes. Increases in 
antibiotic-resistant organisms are of concern because 
they suggest that antibiotics are becoming increas- 
ingly ineffective against certain diseases. 

In addition to HAIs that are infectious diseases, 
there are other infections to which residents are 
susceptible, including skin infections following 
skin breakdowns, such as infected bed sores, and 
urinary tract infections. Figure 3 provides some 
background on these infections. 

In the 2008/09 fiscal year, the Ministry of 
Health and Long-Term Care (Ministry), through 
the Local Health Integration Networks, provided 
funding to long-term-care homes of $2.8 billion. 
This funding only covers a portion of the total 
costs; therefore, residents also pay between about 
$1,600 and $2,200 a month for their accommoda- 
tions, depending on whether they occupy a basic, 
semi-private, or private room. (Private “seniors’ 
residences” such as retirement homes may charge 
more, and do not receive government funding.) 
Because infection-prevention-and-control activities 
should be thoroughly integrated throughout the 
homes’ operations, it can be difficult to identify the 
specific costs of infection prevention and control. 
None of the homes we visited separately tracked 
the costs of preventing and controlling infections. 


Audit Objective and Scope 





The objective of our audit was to assess whether 
selected long-term-care homes followed effective 
policies and procedures for the prevention and 
control of infections. 

Our audit work was conducted at three long- 
term-care homes of different types and sizes that 
provide services to a variety of communities: Exten- 
dicare York (a 288-bed for-profit nursing home 
in Sudbury); Nisbet Lodge (a 103-bed charitable 
home in Toronto); and Regency Manor (a 60-bed 
for-profit nursing home in Port Hope). All three 
homes comply with the structural requirements 
the Ministry set in 1972 for such criteria as size of 
rooms and number of beds per room. Our work 
excluded municipally run long-term-care homes 
because the Auditor General Act does not apply to 
grants to municipalities (other than permitting 
the Auditor General to examine a municipality’s 
accounting records to determine whether a grant 
was spent for the purposes intended). 

In conducting our audit, we reviewed relevant 
files and administrative policies and procedures, 
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and met with appropriate staff of long-term-care 
homes and the Ministry. We obtained the perspec- 
tive of the Ontario Long-Term Care Association and 
the Ontario Association of Non-Profit Homes and 





Services for Seniors, which between them represent 
the majority of long-term-care homes in Ontario. We 
discussed the prevention and control of infections 

in long-term-care homes with the Regional Infec- 
tion Control Networks (RICNs), the Local Health 
Integration Networks (LHINs), and the Local Public 
Health Units associated with the three homes we 
visited. We reviewed relevant research from other 
jurisdictions, including best practices for the preven- 
tion and control of HAIs, such as those issued by the 
Society for Healthcare Epidemiology of America and 
the Association for Professionals in Infection Control 
and Epidemiology, Inc. In addition, we engaged on 
an advisory basis two independent consultants, who 
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Figure 3: Two Types of Infections for Which Residents of Long-term-care Homes Are at Risk 


Prepared by the Office of the Auditor General of Ontario 









Description 


skin infections ° an area of skin * being bedridden 
following skin breaks down or ° age-related 
breakdowns/ develops an open 


pressure ulcers wound as a result of, 


condition 


(such as infected for example: 
bed sores) * prolonged pressure 
¢ friction 


* bacteria residing on _ 
the skin infect the 
open wound 


Selected Risk Factors 


deterioration of skin 







Selected Best 
Practices for 
Prevention 


Examples of 
Possible Effects 








* pain, pus, and redness at 
infected site 


* repositioning to 
minimize ongoing 
° sepsis (infection spreads pressure to any one 
to blood, with symptoms area 
including fever, chills, * assessing skin (upon 
low blood pressure, and admission and then 
changes in mental status) quarterly) 
* death in extreme cases * good nutrition to 
minimize age-related 
deterioration of skin 
condition 


Saas 


urinary tract 
infections 


° bacteria originating 
from the digestive 
tract enter the 
urethra (the tube that 
carries urine from the 
bladder to outside 
the body), multiply, 
and infect the urinary 
tract (including the 
kidneys and bladder) 


bladder 


° being incontinent 
* having a catheter 
inserted to drain 

urine from the 


* strong urge to urinate * minimizing catheter 


sharp pain or burning © USE 
sensation when urinating ° using catheters 
© sepsis (infection spreads appropriately 
to blood, with symptoms 
including fever, chills, 
low blood pressure, and 
changes in mental status) 


¢ death in extreme cases 





have expert knowledge of infection prevention and 
control in long-term-care homes, to assist us. 

We examined the Ministry’s inspection reports 
and other reports as they related to infection pre- 
vention and control in the homes we visited, but did 
not review the Ministry’s inspection process in depth 
because the Office of the Ombudsman of Ontario 
was conducting a review of this process at the time 
of our audit and the Ministry was redesigning its 
inspection process. 

We compared the infection-prevention-and- 
control processes in place at the homes we visited 
against the best practices for infection prevention 
and control for long-term-care homes developed 
by the Provincial Infectious Diseases Advisory 
Committee (PIDAC). The best-practice documents 
that PIDAC has produced reflect recommenda- 
tions made by various organizations, including the 
Public Health Agency of Canada and the College 
of Physicians and Surgeons of Ontario, as well as 
other best practices. In addition, we discussed the 
management of infection prevention and control 


in long-term-care homes directly with members of 
PIDAC and also considered legislative and ministry 
policy requirements. 

Our audit focused on C. difficile, FRI, MRSA, 
VRE, and the prevention of urinary tract infections 
and skin breakdowns (such as bed sores) that can 
become infected. We selected these HAIs primarily 
due to their potential negative impact on resident 
health. We specifically selected MRSA and VRE 
which, because they are antibiotic-resistant organ- 
isms, may be more difficult to treat when they cause 
infections (because few antibiotics are available 
to treat them), and have significantly increased in 
prevalence in recent years. In addition, we looked 
at C. difficile because of the reported widespread 
use of antibiotics in long-term-care homes, which 
increases the chance of developing C. difficile. 

We did not rely on the Ministry’s internal audit 
service team to reduce the extent of our audit work 
because it had not recently conducted any audit 
work on infection prevention and control in long- 
term-care homes. None of the homes we visited had 
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an internal audit function, although all the homes 
conducted some procedures, which we reviewed, 
to help verify whether selected prevention-and- 
control processes were followed. 


Summary 





An estimated 30% to 50% of health-care-associated 
infections (HAIs) are preventable in acute-care 
institutions that have effective infection-preven- 
tion-and-control processes in place. Although little 
information exists concerning HAIs in long-term- 
care homes, it should be recognized that homes’ 
main challenges in preventing and controlling 
infections tend to be somewhat different than 
those of hospitals. For instance, homes have a very 
limited ability to isolate residents with an infectious 
disease, cognitively impaired residents tend to wan- 
der, and staff often have limited HAI training. To 
address these challenges, all three homes we visited 
had a number of formal and informal processes in 
place to prevent and control HAIs. For example, 
most residents at the homes were immunized annu- 
ally against influenza, the homes had established 
specific cleaning schedules for residents’ rooms, 
and the homes collected and reviewed information 
on the cases of certain infections. However, as the 
following observations indicate, there is room for 
improvement in a number of areas: 

e Although the Ministry of Health and Long- 
Term Care (Ministry) has introduced a num- 
ber of initiatives to help prevent and control 
infectious diseases in long-term-care homes, 
it does not have information on the total 
number of cases of most HAIs in long-term- 
care homes. The information collected at the 
homes we visited was generally not compar- 
able because the homes defined and counted 
HAIs in different ways. 

e Although the three homes all had policies to 
screen new residents for febrile respiratory 
illnesses (FRIs), such as influenza, documen- 


tation at two of the homes indicated that just 
60% to 80% of new residents sampled were 
screened. At the third home, there was no 
evidence of formal screening for FRIs. 

Each home had a policy to test new residents 
for tuberculosis (TB) within 14 days of admis- 
sion, as required by legislation. One home 
tested all new residents in our sample, but the 
other two homes tested only 70% and 80% 

of new residents in our sample, respectively. 
Further, when testing was performed, in some 
cases it did not take place until 60 to 125 days 
after a resident’s admission to the home. 
Hand hygiene is the most important activ- 

ity for controlling the spread of infectious 
diseases. Each home recently conducted its 
first review of staff compliance with certain 
hand-hygiene policies. (C. difficile, MRSA, and 
VRE are most commonly spread via the hands 
of health-care workers.) 

Homes generally did not have unoccupied 
rooms to move infectious residents into, and 
indicated that it is disruptive to move other 
residents (some of whom may have paid a 
premium for a private or semi-private room) 
out of their rooms with their personal belong- 
ings. According to Ontario’s Provincial Infec- 
tious Diseases Advisory Committee (PIDAC), 
residents with an FRI who share a room 
should have the curtain drawn around their 
bed. However, all three homes indicated they 
would only pull a curtain around a resident’s 
bed if the resident requested it. 

Although two of the homes had policies to 
clean all touched surfaces in residents’ rooms 
daily, in accordance with PIDAC’s recommen- 
dations, the third home’s policy was unclear; 
but the home indicated that all touched 
surfaces in residents’ rooms are cleaned daily. 
Although PIDAC recommends cleaning the 
rooms of residents who have C. difficile twice a 
day, none of the homes did this. 

None of the homes had processes in place, 
such as sign-off sheets, to record whether 
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residents unable to reposition themselves 
were repositioned every two hours in accord- 
ance with the home’s policies to prevent skin 
breakdowns (such as bed sores). 

e In the 2008/09 fiscal year, 81 C. difficile 
outbreaks in homes were reported to the 
Ministry. We noted that the judicious use 
of antibiotics has been shown to reduce the 
incidence of C. difficile. However, although all 
the homes’ contracted pharmacies provided at 
least some information on antibiotic use, none 
of the homes had a formulary that lists the 
antibiotics physicians can prescribe, as recom- 
mended by PIDAC. 

@ Unlike hospitals, long-term-care homes are 
not required to report publicly on certain 
patient-safety indicators, such as health-care- 
acquired cases of C. difficile, MRSA and VRE, 
as well as hand-hygiene compliance among 
health-care workers. One of the homes we 
visited posted certain infection rates publicly, 
but the other two homes did not. 

@ All three homes had designated an Infection 
Prevention and Control Professional (ICP), in 
accordance with ministry requirements, but 
none of the ICPs had specific training in infec- 
tion prevention and control as recommended 
by PIDAG, and they were all performing this 
role in addition to various other functions. 

We would like to acknowledge the good co- 
operation we received from the long-term-care 
homes we visited. 

We sent this report to the homes we visited and 
the Ministry, and invited them to provide an overall 
response. To be succinct and avoid repetition, we 
summarized the overall responses we received 
from the homes below, followed by the Ministry’s 
overall response. We also summarized the homes’ 
responses to specific recommendations following 
each recommendation and included the Ministry’s 
responses, if applicable. 


The health, safety, and well-being of residents 
of Ontario’s long-term-care homes are of 
paramount importance to the Ministry. In 
considering any aspect of care provided in 
these homes, it is important to note that they 
are primarily the “home” of their residents. The 
Ministry requires homes to comply with legisla- 
tion and regulations as well as standards and 
criteria set out in policy and service agreements. 
Homes are currently required to have an infec- 
tion-prevention-and-control program, which 
includes ongoing surveillance to determine the 
presence of infections and the provision of train- 
ing to all staff. The Ministry has sent a letter to 
all homes reminding them of their obligation to 
meet these standards. 

To improve care for residents, the new 
Long-Term Care Homes Act, 2007, which will 
be proclaimed into force when its regulations 
are finalized, requires that all homes have an 
infection-prevention-and-control program. 
Draft regulations released for public consulta- 
tion in May 2009 include provisions on various 
infection-prevention-and-control measures. 
These regulations enhance current require- 
ments and are consistent with certain key 
recommendations made by the Provincial Infec- 
tious Diseases Advisory Committee (PIDAC) 
that are relevant to long-term-care homes. The 
Ministry encourages homes to implement best 
practices recommended by PIDAC to the extent 
they are able. During winter 2010, the Ministry 
will engage the Regional Infection Control Net- 
works, homes, and other stakeholders in discus- 
sions on how best to meet the recommendations 
in PIDAC’s August 2009 best-practice document 
on Routine Practices and Additional Precautions 
in all health-care settings. 

In consultation with its sector partners and 
stakeholders, the Ministry is also implementing 
a number of initiatives, including a Pressure 
Ulcer Awareness project, the adaption of the 


Infection Prevention and Control at Long-term-care Homes \ 165 


“Just Clean Your Hands” program for use in 
long-term-care homes, and a computerized 
care-management system that will help health 
professionals in homes assess and monitor the 
care needs of residents. As well, the Ministry 
assembled a joint Task Force on Medication 
Management that examined issues related to 
medication-management safety in long-term- 
care homes and their impact on the quality of 
care and life of residents. 





Overall, the homes generally agreed with our 


recommendations but expressed concerns in 
some areas that limited financial and human 
resources may affect their implementation. One 
home highlighted that its ability to implement 
the recommendations is limited by its role in the 
health-care system because, unlike a hospital, it 
provides a home for its residents to live in and 
therefore has unique infection-prevention-and- 
control challenges. 


Detailed Audit Observations 





ROLES AND RESPONSIBILITIES FOR 
INFECTION PREVENTION AND CONTROL 


Long-term-care homes are licensed or approved by 
the Ministry of Health and Long-Term Care (Min- 
istry) under three different laws: the Nursing Homes 
Act, the Charitable Institutions Act, and the Homes 
for the Aged and Rest Homes Act. These three acts do 
not have identical requirements regarding infection 
prevention and control, but the ministry policies set 
out in the Long-Term Care Homes Program Manual 
apply to all types of homes. All three acts and the 
manual will be replaced by the Long-Term Care 
Homes Act, 2007, which received royal assent in 
June 2007 and will be proclaimed into force when 
its regulations are finalized. 


The Ministry is responsible for setting stan- 
dards of care and conducting inspections of long- 
term-care homes. The Ministry conducts annual 
unannounced inspections of all homes to monitor 
compliance with legislation and ministry policies, 
among other things. This includes monitoring cer- 
tain aspects of infection prevention and control. The 
number of unmet criteria noted during the inspec- 
tion is publicly reported on the Ministry’s website. 

Homes are responsible to adopt, follow, and 
monitor effective infection-prevention-and-control 
policies and procedures. Physicians and nurses 
working in the homes have professional responsibil- 
ities related to infection prevention and control, as 
set out in standards and guidelines published by 
their respective regulatory colleges. Further, other 
home staff, including personal support workers and 
cleaning staff, and residents themselves, their fam- 
ilies, and other visitors, all play a role in preventing 
and controlling the spread of infections in homes. 
So, too, do other organizations, including Regional 
Infection Control Networks and local public health 
units, as shown in Figure 4. 


INITIATIVES AND BEST PRACTICES 

FOR PREVENTING AND CONTROLLING 
INFECTIONS ACQUIRED IN LONG-TERM- 
CARE HOMES 


Ministry Initiatives 


A number of initiatives for preventing and control- 
ling infections arose from the outbreak of severe 
acute respiratory syndrome (SARS) in Ontario and 
other parts of the world in 2003. Key among these 
were the Ministry’s establishment of a Provincial 
Infectious Diseases Advisory Committee (PIDAC) 
and Regional Infection Control Networks (RICNs). 
PIDAC was established as part of Operation 
Health Protection, a three-year plan that the 
Ministry issued to revitalize the public-health 
system in Ontario, following recommendations 
from reports written in response to SARS. PIDAC 
is a multi-disciplinary scientific body that provides 
evidence-based advice regarding multiple aspects 
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Figure 4: Selected Key Roles and Responsibilities for Infection Prevention and Control in Long-term-care Homes 
Prepared by the Office of the Auditor General of Ontario 


MINISTRY OF HEALTH AND LONG-TERM CARE 






* cost shares with 
municipalities 

° sets Ontario Public 

Health standards and 

supports adoption of 

these standards 













° sets standards of care 
* conducts inspections 






¢ funds and supports 










* cost share with province 
and administer 














14 Local Health Integration 
Networks (LHINs) 















° assist homes to prevent 
and control infectious 
diseases 

* co-ordinate activities 

during an outbreak 









° provide information and 
education on infection 
prevention and control 

* co-ordinate and promote 

related regional activities 








¢ fund homes 


* co-ordinate regional health 
care, including homes 









LONG-TERM-CARE HOMES 


° set and follow effective policies 
and procedures to prevent and 
control infections 

















* take preventive measures 
(such as hand washing) 


° fulfill professional and 
other responsibilities 















_ Health-care Providers 


(such as doctors and nurses) Ss ipa coe ae - 
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of infectious-disease identification, prevention, and 
control to Ontario’s Chief Medical Officer of Health. 
PIDAC’s work on preventing and controlling infec- 
tions includes: 

@ issuing a number of best-practice documents 
that incorporate applicable guidelines and 
recommendations from entities such as the 
Public Health Agency of Canada and the Col- 
lege of Physicians and Surgeons of Ontario, 
as well as recommendations from medical 
literature (see the Appendix for a list of such 
documents); and 

@ in conjunction with the Ministry, developing 
educational material to enhance infection- 
control training for front-line staff (see Core 
Competencies Projects in the Appendix for 
examples). 

Fourteen RICNs have been established through- 
out Ontario (one in each LHIN), one of which was 
still in the start-up phase at the time of our audit. 
The RICNs are to co-ordinate infection-prevention- 
and-control activities and promote standardization 
in health-care facilities across Ontario. In 2008, the 
RICNs issued the results of a province-wide survey 
of different health-care settings, including long- 
term-care homes, conducted to identify and evalu- 
ate infection-control resources. As well, a number 
of the RICNs have undertaken various initiatives 
pertaining to infection prevention and control in 
homes, including hosting educational sessions and 
developing and disseminating resource material. 

At the time of our audit, the Ministry was in the 
process of adapting the program materials for its 
“Just Clean Your Hands” Hand Hygiene Improve- 


ment Program for use in homes (see the Appendix 
for details). It was also providing training and other 
support for “Stop! Clean Your Hands,” a collabora- 
tive effort between the Canadian Patient Safety 
Institute, the Community and Hospital Infection 
Control Association—Canada, Accreditation Can- 
ada, and the Public Health Agency of Canada. 

As well, at the time of our audit, the Ministry 
was preparing for the new Long-Term Care Homes 
Act, 2007, which is to come into force once its 
regulations are finalized. The Ministry told us that, 
as part of the development of these regulations, it is 
redesigning its inspection processes for long-term- 
care homes to be more risk-based and will include 
several risk indicators related to infection control, 
such as the prevalence of pneumonia and urinary 
tract infections. 


Best Practices 


Although there is minimal data available concern- 
ing HAIs in long-term-care homes, PIDAC has noted 
that an estimated 30% to 50% of health-care- 
associated infections in acute-care facilities, such 
as hospitals, are preventable. Some of PIDAC’s key 
best practices, as outlined in the documents listed 
in the Appendix, are shown in Figure 5. PIDAC 

has also stated that an infection-prevention-and- 
control program that is effective in preventing 
health-care-associated infections can substantially 
reduce health-care costs. More importantly, such a 
program can also substantially reduce the morbid- 
ity (disease) and mortality (death) associated with 
these infections. 





Figure 5: Selected Best Practices for Preventing and Controlling Health-care-associated Infections 
Source of data: Publications of the Provincial Infectious Diseases Advisory Committee 





screening: to identify residents with MRSA, VRE, and FRI 


routine practices and infection-specific precautions: proper hand hygiene; proper cleaning of resident rooms; use of personal 
protective equipment—such as gloves, long-sleeved gowns, and face masks—when appropriate; placement of residents in 


private rooms when appropriate 


immunization: Immunization of residents and staff to prevent the acquisition of communicable diseases 
antibiotic use: the judicious use of antibiotics to reduce resident susceptibility to certain infectious diseases and help prevent 


infectious diseases that are antibiotic-resistant 


surveillance: tracking and analyzing infection data in order to take timely corrective action 
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Accreditation 


The Ministry recognizes two accreditation organ- 
izations for long-term-care homes: Accreditation 
Canada and the Commission on Accreditation of 
Rehabilitation Facilities. At the time of our audit, 
the three homes we visited were accredited by 
Accreditation Canada. Accreditation Canada 
examines the quality of health services at homes 
with the aim of helping them improve the quality of 
service they provide to residents. The accreditation 
process includes reviewing organizational practices 
pertaining to infection control, such as whether the 
home tracks infection rates and whether the home 
delivers education regarding hand hygiene. 


SCREENING 


Screening generally enables homes to identify 
newly admitted residents who have an infectious 
organism or disease, and to implement certain 
additional measures and precautions, if needed. 
Screening generally involves considering various 
factors to identify which residents have symptoms 
of an infectious disease or have a higher risk of 
having acquired certain organisms or diseases. 
Samples, where appropriate, are then taken from 
these residents and forwarded to a laboratory, 
which determines whether the residents have the 
organism or disease. In some cases, a home will 
extend screening to every resident admitted. This is 
called “universal screening.” 

In its best-practice documents, PIDAC notes that 
screening is an important step in keeping an infec- 
tious organism or disease from spreading to other 
residents, staff, and visitors. PIDAC recommends 
that homes should: 

@ assess all residents being admitted or readmit- 
ted for symptoms of FRI, such as cough, 
shortness of breath, and fever. Homes are 
encouraged to take an “active” approach to 
this screening; for example, staff should ask 
residents about possible symptoms and take 
into account whether the residents have been 


in any contact with others that might have put 
them at risk. Homes may also use “passive” 
screening, such as posting signs requesting 
that residents who have FRI symptoms notify 
staff. 

@ actively screen all residents being admitted 
or readmitted to determine their risk of hav- 
ing MRSA or VRE. Staff should ask, among 
other questions, whether the resident has 
previously had MRSA or VRE; if he or she has 
been admitted to or has spent more than 12 
continuous hours as a patient in any health- 
care facility, such as a hospital, in the past 12 
months; and if he or she has been recently 
exposed to a health-care-facility unit with 
a MRSA or VRE outbreak. A “yes” to any of 
these questions makes a resident high risk, 
and homes should take a sample from such 
residents to determine if they actually have 
MRSA or VRE. 

@ regularly conduct audits to evaluate their 
screening practices as part of a continuous 
program for managing and improving quality. 

Legislation also generally requires homes to 

screen all new residents for tuberculosis within 14 
days of admission unless the resident was tested in 
the last 12 months. 


Respiratory Illnesses 


We found that the three homes we visited all had 
policies requiring new residents to be screened 
for FRIs on admission, as well as for tuberculosis 
within 14 days of admission. In addition, two of 
the homes used a checklist to document when FRI 
screening was complete. 

For FRIs, our sample of new residents at the two 
homes that used checklists indicated that, at one 
home, there was documentation that 80% of new 
residents were screened and, at the other, 60% of 
new residents were screened. At the third home, 
there was no evidence of formal screening for FRIs; 
however, the home informed us that all new resi- 
dents were informally screened. 
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With respect to tuberculosis, our sample 
indicated that one home had screened all new resi- 
dents, while the other two only screened 70% and 
80%. Where screening was done, it was completed 
more than 14 days after admission for 29% of new 
residents sampled at one home; for 40% at the 
second home; and for 75% at the third home. We 
noted cases in which screening did not take place 
until up to 60 days after admission at two homes 
and 125 days at the third. 

Except for one home’s evaluation of its tuber- 
culosis-screening practices and another home’s 
review of five resident files, none of the three 
homes had conducted any formal review of their 
screening practices for respiratory illness in 2007 
or 2008. The home that had formally evaluated 
its tuberculosis screening practices noted that, in 
2007, only 53% of new residents sampled had been 
screened for tuberculosis as required. As a result, 
in December 2008, the home began monitoring 
whether all required steps are completed whenever 
a new resident is admitted, including screening for 
tuberculosis. 


Methicillin-resistant Staphylococcus 
aureus (MRSA) and Vancomycin-resistant 
Enterococci (VRE) 


Although there is little authoritative guidance on 
when universal screening is appropriate, two of 

the three homes we visited had policies to screen 
all residents for MRSA and VRE on admission and 
readmission, such as from a hospital. The third 
home did not screen new or returning residents for 
MRSA or VRE; however, in early 2009, it began ask- 
ing its sending facilities, such as hospitals, whether 
residents admitted to the home had MRSA or VRE. 
In addition, at the time of our audit, this home was 
in the process of arranging electronic access to the 
results of hospital tests, such as for MRSA and VRE, 
which its residents have undergone. Doing this may 
reduce the need for additional testing for some resi- 
dents. We noted that the other two homes generally 
did not request similar information. 


We reviewed a sample of new residents and 
readmitted residents for 2008 at the two homes 
with policies to screen for MRSA and VRE. One of 
these homes screened all residents in our sample 
for MRSA, but told us it did not screen any residents 
for VRE because it had not had a case of VRE in 
the last couple of years. At the other home, almost 
two-thirds of residents sampled were not screened 
for MRSA and VRE. Neither of the two homes had 
undertaken any formal reviews to ensure that 
residents admitted or readmitted to the home were 
screened for MRSA and VRE, something that their 
established policies stipulate should be done. 

We also observed that none of the homes visited 
had a policy to screen residents considered to be a 
VRE contact (that is, for example, a resident whose 
roommate has VRE or who has been in physical 
contact with a resident found to have VRE). Only 
one of the homes had a policy to screen residents 
considered to be a MRSA contact. 


| RECOMMENDATION 1 
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To ensure that residents with infectious diseases 
are identified quickly enough to minimize the 
risk of the disease spreading to others, long- 
term-care homes should periodically monitor 
whether their screening processes are in accord- 
ance with the recommendations made by the 
Provincial Infectious Diseases Advisory Commit- 
tee and legislative requirements. 





The homes generally agreed with this recom- 


mendation. One home noted that it will work 
with its Regional Infection Control Network 

to update certain screening processes to be 

in accordance with recommendations from 
the Provincial Infectious Diseases Advisory 
Committee. As well, it will review its screening 
processes quarterly, with summarized results 
reported to its Infection Control Committee 
and Professional Advisory Committee. Results 
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will be tracked electronically and reviewed for 
trends. Where needed, an improvement action 
plan will be put into place. Further, this home 
indicated that it will continue to pursue access 
to hospital electronic records, including lab test 
results, and will request that hospitals provide 
information on any infectious diseases that new 
and returning residents may have. This home 
also indicated that it would work with infection- 
control organizations to identify appropriate 
time intervals after which re-testing for key 
infections, such as MRSA, would be prudent. 
Another home commented that it will estab- 
lish policies concerning the screening of all resi- 
dents re-admitted to its home, and will monitor 
to ensure that all admissions are screened for 
FRIs, MRSA and VRE, and that residents admit- 
ted with diarrhea are screened for C. difficile 
in accordance with its policies. Using informa- 
tion gained from the monitoring process, its 
Continuous Quality Improvement Committee 
will review potential methods to improve the 
home’s processes. This home also indicated that 
because many cases of illness are acquired by 
residents when they are at other health-care 
organizations, such as hospitals, it would be 
beneficial both to the home and system-wide 
if these organizations had a standardized exit 
surveillance-screening process. In particular, 
the organizations should provide homes with 
screening test results, as appropriate, because 
this would assist the homes in controlling the 
spread of infectious diseases. 


RESIDENT CARE 


Routine Practices and Infection-specific 
Precautions 


There are a number of practices that, if always used 
by homes with all their residents during all care, 
can help prevent and control the transmission of 
microorganisms that cause infectious diseases. 


Health Canada and the Public Health Agency of 
Canada call these “routine practices.” Accord- 

ing to PIDAC, only the consistent use of routine 
practices—particularly washing hands before and 
after contact with a resident and the resident’s 
environment—will prevent the spread of infectious 
diseases. PIDAC has also noted that additional 
precautions are necessary to prevent and control 
certain infectious diseases such as MRSA, VRE, 
and C. difficile. With respect to these practices and 
precautions, PIDAC states the following: 

e Hand Hygiene—Before and after contact with 
each resident and the resident’s environment, 
staff must wash their hands with an alcohol- 
based rub (70% to 90% alcohol preferred) 
or soap and water. An alcohol-based rub is 
generally preferred when hands are not vis- 
ibly soiled. However, soap and water may be 
more effective than an alcohol-based rub in 
removing C. difficile spores. All health-care 
settings, including long-term-care homes, 
must develop and implement a hand-hygiene 
program that includes ongoing monitoring 
and observation of hand-hygiene practices. 

e Use of personal protective equipment—When 
entering the room of a resident infected with 
C. difficile, health-care workers must wear 
gloves and gowns. When providing direct care 
to a resident with MRSA or VRE, they must 
wear gloves and should wear gowns. They 
must remove their gloves and gowns before 
exiting the resident’s room. Homes should 
monitor compliance with the recommended 
use of personal protective equipment. 

e Use of private rooms—Long-term-care homes 
should place residents with certain infectious 
diseases and residents suspected of having 
C. difficile in a private room with its own toilet. 
If all the home’s private rooms are occupied, 
infection-prevention-and-control staff should 
be consulted to arrange for residents to share 
a room with similarly infected residents (this 
is known as “cohorting” residents). 
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e Cleaning of resident rooms—All touched 
surfaces in each resident’s room must be 
cleaned daily. As well, homes should take 
special precautions in cleaning the rooms of 
residents having or suspected of having C. dif- 
ficile because this organism has been found 
on surfaces such as door handles and faucets. 
(We understand that PIDAC expects to release 
a best-practice document on environmental 
cleaning in spring 2010.) Disease-specific 
recommendations for C. difficile include: 

e Ifthe resident has or is suspected of having 
C. difficile, homes should clean all horizon- 
tal surfaces in the resident’s room and all 
items within reach of the resident twice 
daily with a hospital-grade disinfectant. 
Staff should pay particular attention to 
cleaning frequently touched areas such as 
bed side-rails, telephones, and toilets. 

e Homes must communicate clearly with 
cleaning staff to ensure that they know 
which rooms require twice-daily cleaning. 

e Homes should develop and use a checklist 
to monitor that cleaning is done twice daily. 

Similarly, Health Canada recommends that 

homes clean resident rooms according to a predeter- 
mined schedule that assigns staff to specific tasks 
for keeping surfaces clean and dust free. This is 
consistent with the Ministry’s Long-Term Care Homes 
Program Manual. As well, PIDAC states that homes 
should conduct periodic audits of their cleaning 
protocols to make sure that they are followed. 


Hand Hygiene 
PIDAC notes that the most common way microor- 
ganisms are transmitted is on the hands of health- 
care providers and that, therefore, hand hygiene 
is the most important activity for controlling the 
spread of infectious diseases. However, PIDAC also 
notes that, despite the importance of hand hygiene, 
compliance with hand-hygiene protocols by health- 
care providers is low. Health Canada has also 
observed that studies have repeatedly documented 





that health-care providers, including resident-care 
staff in long-term-care homes, fail to wash their 
hands. A 2005 study at two Ontario homes noted 
that overall hand-hygiene compliance was less than 
15%. 

Various studies have noted that impediments to 
handwashing include: 

@ lack of time due to, for example, understaffing 

and inaccessibility of sinks; 

@ inadequate supplies for handwashing; 

@ concerns over handwashing products and the 

effects of frequent washing on hands; 

e belief that handwashing is not necessary if 

gloves are used; and 

@ skepticism about the value of washing hands 

when they are not visibly soiled. 

All three of the homes we visited had policies in 
place with respect to hand hygiene that were con- 
sistent with best practices noted by PIDAC, includ- 
ing when hand hygiene should take place, which 
products to use, and appropriate handwashing 
techniques. Also, they had all provided one or two 
hand-hygiene educational sessions to staff between 
January 2008 and March 2009. 

In 2007, one of the homes participated in a pilot 
project for the “Stop! Clean Your Hands” initiative. 
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The home indicated that its involvement included 





an educational session for staff, as well as displaying 
hand-hygiene posters throughout the home. 

At the time of our audit, all three homes had 
recently conducted their first audit to determine 
compliance with their hand-hygiene policies. We 
reviewed the results of the audits and noted that, 
with a few exceptions, homes were reporting 80% 
to 100% compliance. Given Health Canada and 
others’ observations that resident-care staff often 
fail to wash their hands, such unexpectedly good 
results may indicate the need for these homes to 
review their audit-monitoring methodology to 
ensure that it is independent and objective. 

As previously noted, the use of alcohol-based 
hand rub is generally preferred for hand hygiene if 
hands are not visibly soiled. PIDAC states that for 
maximum efficacy, alcohol-based hand rub must 
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be available at the place where staff provide care 
to residents. We noted that one of the homes had 
alcohol-based hand rub available in resident rooms 
and in its dining room and nursing stations. At the 
second home, alcohol-based hand rub was available 
near one sink on each floor, and, starting in spring 
2009, health-care staff were to carry a caddie con- 
taining the hand rub. The third home had alcohol- 
based hand rub available in numerous locations 
including corridors, resident dining rooms, and 
nursing stations. 


Use of Personal Protective Equipment 
All three homes we visited had policies in place 
regarding the use of personal protective equipment, 
such as gloves, gowns, and face masks, which were 
consistent with best practices noted by PIDAC. As 
well, for residents who had infectious diseases, such 
as C. difficile, the homes’ policies were to place signs 
on the doors stating that additional precautions, 
such as wearing gloves and a gown, must be taken 
by anyone entering the room. 

We noted that the three homes reviewed, to 
varying extents, whether their resident-care staff 
wore gloves when appropriate. Two of the homes 
had completed one review of a few resident-care 
staff, while the third home conducted regular 
reviews three times a week in 2008. As well, two 
of the homes reviewed whether resident-care staff 
washed their hands after removing the gloves, and 
one home looked at whether staff washed their 
hands before putting on gloves. Our review of the 
results indicated few problems at two of the homes, 
whereas the third home indicated compliance was 
just over 50%. 

None of the homes had conducted reviews of the 
use of other personal protective equipment, such as 
gowns and masks. 


Use of Private Rooms 
One infection-specific precaution PIDAC recom- 
mends is isolating in private rooms residents who 
have certain infectious diseases. At the homes we 


visited, between about 5% and 49% of residents 
had a private room. When placing residents in 
private rooms is not possible, PIDAC recommends 
cohorting residents with similar infectious diseases. 
All three homes told us that isolating and cohorting 
residents is generally not practical because each 
resident’s room is his or her “home” containing 

his or her own personal belongings. The three 
homes also told us that they generally do not have 
unoccupied rooms, it is disruptive to move residents 
out of their rooms, and many residents have paid a 
premium to be in a private or semi-private room. 

If a resident with C. difficile has to share a 
room or bathroom with other residents, PIDAC 
recommends that the resident be provided with 
a commode chair. Two of the homes had policies 
consistent with PIDAC’s recommendation; the third 
home did not. Only one of the two homes with 
policies to provide commode chairs reported having 
residents with C. difficile in 2008. However, this 
home had not conducted any reviews to determine 
whether staff had provided a commode chair to 
residents with C. difficile. 

Health Canada indicates that homes should 
consider separating residents with an FRI who 
share a room from other residents by at least one 
metre. Further, PIDAC recommends that homes 
should have the curtain drawn between resident 
beds. One of the homes we visited had reviewed the 
distance between residents in four rooms and found 
that these residents were often not adequately 
separated. As well, all three homes indicated that 
curtains are only drawn around a resident at a resi- 
dent’s request; for example, if they want privacy. 


Cleaning of Resident Rooms 

PIDAC indicates that all touched surfaces in each 
resident’s room must be cleaned daily. Health Can- 
ada recommends that homes clean resident rooms 
according to a predetermined schedule that assigns 
staff to specific tasks. 

Two of the three homes we visited had policies 
to clean all touched surfaces in each resident’s 
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room daily. The third home’s policy was unclear, 
but the home indicated that all touched surfaces 
in each resident’s room are cleaned daily. All three 
homes had a schedule of cleaning duties assigned 
to specific staff. As well, two of the homes began 
using microfibre cleaning products, such as micro- 
fibre cleaning cloths and mops, in 2008. Various 
studies indicate that these cleaning products are 
more effective than conventional products at 
removing microorganisms. 

PIDAC’s best practices identify special require- 
ments for cleaning the rooms of residents with 
certain infectious diseases, such as C. difficile. This 
is because C. difficile produces spores that a number 
of chemicals are unable to destroy. Even with the 
right chemicals, applying force to create friction 
is necessary to remove the spores. PIDAC also 
recommends that the rooms of residents who have 
C. difficile be cleaned twice daily, and that when 
the infection clears up, a more thorough clean- 
ing should occur, including throwing away toilet 
brushes and disposable items such as paper towels 
and toilet paper. 

All three homes had policies on cleaning the 
rooms of residents with C. difficile. However, only 
one home’s policy required that the rooms of 
residents with C. difficile be cleaned twice a day. 
Notwithstanding this policy, this home, like the 
other two homes, only cleaned the rooms of resi- 
dents with C. difficile once a day. Two of the homes 
indicated that they inform housekeeping staff when 
a resident becomes better so that his or her room 
will be cleaned more thoroughly. The third home 
did not have processes to notify housekeeping of 
the need for a more thorough cleaning. 

We observed that two of the three homes mon- 
itored their cleaning practices as recommended by 
PIDAC. Both of these homes inspected the cleanli- 
ness of resident rooms on a regular basis and identi- 
fied some areas where cleanliness was deficient, 
including floors, vents, windows, and door handles. 
Both homes indicated that deficiencies were dis- 
cussed with staff and corrective action was taken. 
We noted that the results of these audits were not 


summarized to determine an overall level of cleanli- 
ness in the homes. One of these homes also audited 
the cleanliness of a room whenever it was vacated, 
such as when a resident passed away or moved to 
another room. The third home did not inspect the 
cleanliness of resident rooms on a regular basis. 


Immunization 


PIDAC notes that immunization is one of the most 
effective measures for preventing residents and staff 
from acquiring communicable diseases. PIDAC rec- 
ommends that homes have immunization programs 
for residents that include pneumococcal pneumonia 
immunization and annual influenza immunization. 
PIDAC notes that homes should offer appropriate 
immunization for staff, such as annual influenza 
immunization, which can protect not only staff but 
also residents from acquiring influenza. 

The Ministry set certain target immunization 
rates for residents and staff of long-term-care 
homes up to January 2009. As Figure 6 shows, 
in 2008, the homes we visited were generally 
close to or above the targeted rates for influenza 
immunization of residents and staff. However, all 
three homes were below the targeted pneumococ- 
cal immunization rate for residents. The Ministry 
indicated that it was reviewing the appropriateness 
of developing updated target vaccination rates 
because these targets have not been shown to influ- 
ence immunization rates. 





Figure 6: Target and Actual Immunization Rates at 


Three Long-term-care homes, 2008 


Source of Data: Ministry of Health and Long-Term Care and 
audited long-term-care homes 
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Prevention of Selected Infections 


Research indicates that following best practices 

can help prevent certain other infections, including 
infected skin breakdowns, such as infected bed 
sores, and urinary tract infections. For example, 
repositioning an immobile resident every two hours 
may help prevent skin breakdowns, and minimiz- 
ing catheter use may help prevent urinary tract 
infections. 


Skin Infections Following Skin Breakdowns 
Skin breakdowns, also known as pressure ulcers 
(for example, bedsores), can become infected, caus- 
ing pain and possibly more serious complications, 
or even death in extreme cases. Although little Can- 
adian data is available, U.S. studies have found that 
the prevalence of pressure ulcers ranges from 2% to 
28% of residents in long-term-care facilities. 

The Ministry’s Long-Term Care Homes Program 
Manual requires that homes develop and follow 
policies for managing skin care, including assessing 
each resident’s skin upon admission and quarterly 
thereafter. The Program Manual, as well as other 
best-practice guidelines, also states that people at 
risk for pressure ulcers should be repositioned at 
least every two hours. 

All three homes visited had policies in place 
requiring that each resident’s skin be assessed upon 
admission and quarterly. We examined a sample of 
files of residents with skin breakdowns and noted 
that all residents had a skin assessment completed 
upon admission. However, although one home 
completed all quarterly assessments in 2008, the 
other two completed 73% and 82% of the assess- 
ments respectively. In February 2009, one of these 
homes implemented a checklist to document that 
various items were being performed, including 
quarterly skin assessments. 

All homes indicated that they conducted a 
periodic review to ensure that the skin assessments 
were being performed as required. However, none 
of the homes summarized their reviews to deter- 
mine whether there were any home-wide issues to 
be addressed. 


All three homes had policies on managing skin 
care, including repositioning residents who could 
not reposition themselves at least every two hours. 
One of the homes had developed a sign-off sheet to 
document that a resident had been repositioned. 
However, at the time of our audit, staff at the home 
were generally not using it. Neither of the other 
homes were using a sign-off sheet to document that 
a resident was repositioned. 


Urinary Tract Infections 
Research indicates that urinary tract infections are 
generally the most commonly reported bacterial 
infection in residents. Because these infections 
are particularly associated with incontinence, it is 
important to periodically assess a resident’s contin- 
ence. A practice for preventing these infections is 
to minimize the use of catheters. Although there 
is little authoritative research on other ways to 
prevent urinary tract infections, some sources sug- 
gest that ensuring adequate fluid intake may help in 
preventing these infections. 

All three homes visited had policies for per- 
forming continence assessments within seven days 
of a resident’s admission and quarterly assessments 
thereafter, in accordance with the Ministry’s 
Program Manual. We found that initial continence 
assessments were completed for all residents in our 
sample. However, although one home completed 
almost all the quarterly assessments, the other 
two homes completed less than 75%. In February 
2009, one of the homes we visited began using a 
checklist to document that various items, including 
quarterly continence assessments, were completed 
as required. The other two homes had no formal 
monitoring practices to ensure that the quarterly 
continence assessments were being completed. 

Various studies indicate that, for general good 
health, the recommended minimum daily fluid 
intake for residents is 1,500 millilitres. All three 
homes visited had policies consistent with this and 
monitored the amount of fluid consumed each day 
by each resident. We noted that only 10% to 20% of 
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resident files sampled at two of the homes showed 
that the resident had consumed the recommended 
amount of daily fluid. In contrast, at the third 
home, all resident files sampled showed that the 
residents had consumed at least the recommended 
amount of fluids. 


RECOMMENDATION 2 


In order to better prevent the transmission of 

infectious diseases: 

e long-term-care homes should monitor 
whether prevention best practices (such 

as hand hygiene and the use of personal 

protective equipment) and infection-specific 

precautions (such as twice-daily cleaning of 
rooms of residents who have C. difficile) are 
conducted in accordance with the recom- 
mendations made by the Provincial Infec- 
tious Diseases Advisory Committee (PIDAC) 
and review their monitoring methodology 
to ensure that abnormally high compliance 
rates are reflective of actual practices; 

@ the Ministry of Health and Long-Term Care 
should develop guidance to assist homes 

in determining how best to meet PIDAC’s 

recommendations on isolating and cohorting 

residents who have or are at high risk of 
having infectious diseases, given the limited 
availability of private rooms; and 

e long-term-care homes should continue to 
promote and monitor the immunization of 
residents and staff. 

To help prevent residents from acquiring an 
infected skin breakdown, long-term-care homes 
should adopt processes, such as using a sign-off 
sheet for recording when residents are repos- 
itioned, to enable supervisory staff to monitor 
compliance with established procedures. 


The new Long-Term Care Homes Act, 2007, 
which will be proclaimed into force when its 





regulations are finalized, requires that all homes 
have an infection-prevention-and-control pro- 
gram that includes daily monitoring to detect 
the presence of infection in residents as well as 
measures that prevent the transmission of infec- 
tions. Draft regulations on infection prevention 
and control released for public consultation 

in May 2009 include provisions on various 
measures including hand-hygiene programs, the 
availability of personal protective equipment, 
and immunization. 

The Ministry plans to redevelop 35,000 beds 
in older long-term-care homes over the next dec- 
ade to ensure more equitable access to quality 
accommodation. These beds will be redeveloped 
according to structural requirements detailed in 
the Ministry’s Long-Term Care Facility Design 
Manual (May 2009) which provides for larger 
rooms that have a maximum of two beds and 
that all have wheelchair-accessible washrooms. 
This will assist homes in keeping residents with 
infectious diseases adequately separated. 





The homes generally supported this recom- 


mendation. One home indicated that it will 
develop an objective tool to monitor compliance 
with various best practices recommended by the 
Provincial Infectious Diseases Advisory Com- 
mittee (PIDAC), including hand hygiene and the 
use of personal protective equipment, and will 
develop a plan to address any items arising from 
this monitoring. The home’s Infection Control 
Committee and Professional Advisory Commit- 
tee will review summarized results twice a year 
and provide direction. In addition, to increase 
hand hygiene compliance, the home will install 
moisture stations to protect staff's hands from 
over-washing and will use new technologies to 
enhance staff education, such as a substance 
applied to the hands that visually demonstrates 
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the contamination left behind if hands are not 
washed properly. This home also noted that it 
will institute room cleaning checklists to ensure 
that rooms are cleaned in accordance with 
PIDAC recommendations, especially the rooms 
of residents with C. difficile. The checklists will 
be placed on housekeeping carts and will be 
completed by staff as each room is cleaned, 
thereby enhancing data accuracy and enabling 
supervisors to access this information quickly. 
Another home indicated that it will continue to 
improve its strategies to prevent the transmis- 
sion of infections, including following the best 
practices recommended by PIDAC. However, 
the home commented that it is constrained 
by its funding: it would need additional staff 
because existing staff do not have time available 
to take on more responsibilities. The home also 
recognized the need to validate the results of its 
review of hand hygiene. It will continue to mon- 
itor the use of personal protective equipment 
and cleaning activities on an informal basis and, 
if risk indicators demonstrate a need for change, 
will establish a formalized process. Nonetheless, 
the home indicated that it will review its pro- 
cedures and routines, within the limitations of 
its human resources, with the goal of enabling 
twice daily cleaning of rooms with residents 
who have C. difficile. 

Two homes highlighted a number of the 
challenges regarding isolating or cohorting resi- 


dents who have or are at a high risk of having an. 


infectious disease, including the impracticality 
of moving a resident’s furniture and belong- 
ings, and the possible traumatic impact a move 
may have on frail residents and residents with 
dementia. One of these homes noted that put- 
ting an infectious person in a private room will 
delay the admission of another resident and 
result in lost income for the home. However, this 
home suggested that having an infirmary in the 
home, where infectious residents could be tem- 
porarily moved, would be one way of helping 
prevent the spread of certain infectious diseases. 


With respect to promoting and monitoring 
the immunization of residents and staff, one 
home noted that it will continue to promote 
the importance of immunization for residents 
and staff, as recommended by its local public 
health unit and other regulatory authorities. 
Another home indicated that it will develop an 
immunization strategy to further promote the 
immunization of residents and staff, which will 
outline the location and optimal time for staff 
vaccinations. This home is exploring the use of 
automatic reminders for staff when it is time for 
their next immunization. Further, committees in 
this home will continue to monitor the immun- 
ization rates of residents and staff over time, 
and will put in place an improvement action 
plan, if required. 

Regarding the use of a sign-off sheet when 
repositioning residents who are unable to repos- 
ition themselves, one home indicated that it has 
implemented “point of care” electronic tablets 
at nursing stations and near residents’ rooms 
to enable more accurate tracking of certain 
information including pressure-ulcer-prevention 
activities. This home noted that it will continue 
identifying residents with a high risk of a 
skin breakdown and developing strategies to 
prevent these, such as by using pressure-relief 
mattresses. In addition, the home will continue 
to track and analyze pressure ulcer rates, and 
benchmark with other homes and with industry- 
wide benchmarking through participation in 
the Canadian Institute for Health Information’s 
long-term-care quality indicator database. 
Another home worried about the potential 
time involved to document when residents are 
repositioned and therefore supported the use 
of sign-off sheets only for residents with a high 
risk of a skin breakdown or pressure ulcer. The 
third home also implemented the use of sign-off 
sheets for high-risk residents. This home com- 
mented that, in addition to the sign-off sheets, 
informal monitoring hy supervisors enables it to 
verify residents’ well-being. 
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ANTIBIOTIC USE 


Residents in long-term-care homes use antibiotics 
primarily to treat infections. However, infectious 
bacteria are becoming resistant to antibiotics, 
which is increasing the risk that antibiotics will 

no longer effectively treat certain infections in the 
future. In fact, certain bacteria that cause infections 
have become resistant to the preferred antibiotics 
for their treatment. 

Research indicates that there is an association 
between a person’s increased use of antibiotics and 
the resistance of infections to certain antibiotics. 

In addition, individuals are at increased risk for 
acquiring certain infections, such as C. difficile and 
MRSA, if they are taking antibiotics. As mentioned 
in Figure 2, C. difficile infection usually occurs when 
the use of antibiotics reduces the normal levels of 
good bacteria found in the intestines and colon of a 
resident. This reduction in good bacteria allows the 
C. difficile bacteria to grow and produce toxins that 
make the resident sick. Because of this risk, the U.S. 
Food and Drug Administration revised the safety 
warnings for certain antibiotics in June 2007. The 
warnings now indicate that taking the antibiotic 
poses a risk of C. difficile and that nearly all anti- 
biotics have been associated with an increased risk 
of C. difficile. 

Unlike hospitals, long-term-care homes are not 
required to identify outbreaks of C. difficile to their 
local public health unit. They are also not required 
to report outbreaks of C. difficile to the Ministry, 
although quite a few of them do. In 2008/09, 81 
C. difficile outbreaks were reported to the Ministry. 
However, the reported information did not include 
the total number of residents who acquired C. dif- 
ficile during these outbreaks or the resident out- 
comes (for example, deaths). 

The fact that there have been a number of C. dif- 
ficile outbreaks in long-term-care homes in Ontario 
reinforces the need for the judicious use of antibiot- 
ics. Further, medical research indicates that anti- 
biotics are frequently prescribed in long-term-care 
homes, with one study of Canadian and U.S. homes 





identifying that antibiotics were prescribed to 79% 
of residents over a one-year period. 

PIDAC’s recommendations to limit the increase 
and spread of antibiotic-resistant infections include 
that homes should: 

@ develop an “antibiotic stewardship program” 
by implementing policies and procedures to 
promote judicious antibiotic use—one policy 
should be that homes have a drug formulary 
that lists the antibiotics physicians can pre- 
scribe; and 

@ review actual antibiotic use to assess prescrib- 
ing appropriateness. 

None of the three homes we visited had imple- 
mented the recommended antibiotic drug formu- 
lary. However, the physicians we spoke to at these 
homes indicated that they do try to minimize the 
use of antibiotics. 

All three homes we visited had a process in place 
to monitor antibiotic usage to some extent. At one 
home, the pharmacy periodically provided informa- 
tion to the home’s physician on the use of particular 
drugs, including certain antibiotics. The other two 
homes received certain information on antibiotic 
usage directly from their pharmacy: 

@ One home received information on the use of 
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specific antibiotics overall and by physician. 

@ The other home periodically received infor- 
mation on the use of specific antibiotics 
overall, as well as on the total number and 
percentage of residents taking antibiotics. It 
also received a comparison of the percentage 
of residents on antibiotics relative to the other 
long-term-care homes that the pharmacy 
services (approximately 135 homes and 
17,000 beds). No information was received on 
antibiotic-prescribing patterns by physician. 

These two homes indicated that their profes- 

sional advisory committee, which included, among 
others, the home’s administrator, director of care, 
and physician, reviewed the information provided 
by the pharmacy. These reviews were informal and 
not documented. 
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None of the three homes had overall summary 
information on the reasons why certain antibiotics 
were prescribed, and therefore they could not ana- 
lyze antibiotic use patterns. Although each home 
indicated that either its physician or professional 
advisory committee examined the reports from its 
pharmacy, to be able to fully evaluate the informa- 
tion would require a labour-intensive review of resi- 
dents’ health records to determine the reason for 
the drug use. All three homes relied primarily on 
their physicians—who prescribe the antibiotics— 
to tell them whether any changes were required 
regarding the use of antibiotics. 

In our 2007 Annual Report, we noted that the 
Beers Criteria lists certain high-risk drugs that 
experts have indicated are generally more harm- 
ful than beneficial to older adults. At that time we 
found, however, (using information from the Min- 
istry’s Ontario Drug Benefit Program), that at least 
20% of residents in 30 homes were dispensed at 
least one of the eight high-risk drugs in our sample. 
While acknowledging that there may be situations 
where the use of these drugs is warranted, given 
the higher level of usage of the selected Beers 
Criteria drugs we detected in certain homes at that 
time, we recommended in our 2007 Annual Report 
that the Ministry, in conjunction with the College 
of Physicians and Surgeons of Ontario, periodically 
review the use of higher-risk drugs. Our current 
audit did not review the use of Beers Criteria drugs. 
However, with respect to antibiotic use, we believe 
it would be beneficial for the Ministry, in conjunc- 
tion with the College of Physicians and Surgeons of 
Ontario, to perform a similar periodic review of the 
use of antibiotics in long-term-care homes. 


RECOMMENDATION 3 





To help prevent antibiotic-resistant organisms 
and reduce the susceptibility of residents to cer- 
tain infections, the Ministry of Health and Long- 
Term Care, in conjunction with other interested 
stakeholders, should: 


@ assist long-term-care homes to develop a 
drug formulary; and 

@ periodically review the use of antibiotics 
in long-term-care homes so that follow-up 
action can be taken where the use of anti- 


biotics seems unusually high. 





The Ontario Drug Benefit Act and the Drug 
Interchangeability and Dispensing Fee Act govern 
the administration of Ontario’s public drug 
programs. The Ministry maintains and publishes 
the Ontario Drug Benefit Formulary/Compara- 
tive Drug Index (Formulary), which identifies 
all the drug products that eligible individuals, 
such as residents in long-term-care homes, may 
receive under these programs. The Formulary 
is to be used by prescribers and pharmacists 

as a guide for prescribing and reimbursement. 
Creating a separate formulary for antibiotic 

use in homes would lead to duplication of the 
process and may result in undue confusion 

for physicians and pharmacists. However, the 
Ministry is supportive of each home developing 
internal policies based on evidence-based infor- 
mation to promote best practices for antibiotic 
use. This is based on the Ministry’s recognition 
that decisions relating to the use of antibiotics 
are generally the responsibility of a resident’s 
physician. The Ministry relies on the physician’s 
professional judgment in deciding which 
antibiotic to prescribe based on each resident’s 
unique medical history. 

It is the mandate of the physician’s and other 
drug prescriber’s professional regulatory col- 
leges to educate, direct, and, potentially, sanc- 
tion prescribers regarding prescribing practices. 
Also, many pharmacies contracted by homes 
review medication-use patterns and trends, 
including those for antibiotics, to identify issues, 
and will work with the home as appropriate. 

In addition, an interdisciplinary team at each 
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home is responsible for reviewing residents’ 
charts to determine, among other things, the 
patterns and appropriateness of antibiotic use. 
Furthermore, the Ministry received the Joint 
Task Force on Medication Management in Long- 
Term Care’s report in fall 2009, which examined 
issues relating to medication-management 
safety and was reviewing its recommendations. 
The report and its recommendations is to be 
shared with stakeholders. 





Although the homes were not requested to 
respond to this recommendation, two of the 
homes highlighted that they rely on their 
physicians’ clinical judgment in prescribing 


medication for residents on a case-by-case 
basis, and that their physicians were aware of 
the risks associated with the medications. One 
home questioned whether a formulary was 
necessary and was concerned that physicians 
might be reluctant to have any restrictions 
placed on what drugs they can prescribe under 
the Ontario Drug Benefit Plan. The other home, 
however, commented that it will present this 
matter to its Professional Advisory Committee, 
which reviews drug utilization, for opportun- 
ities to improve monitoring of antibiotic use that 
would result in value-added data for resident 
care. As well, this home indicated that it would 
work to support Ministry-led initiatives that 
promote the effective and safe use of antibiotics 
for its residents. 


SURVEILLANCE 


PIDAC defines surveillance as the systematic, 
ongoing collection, collation, and analysis of data 
with timely distribution of the information to those 
who require it in order for action to be taken where 
necessary. PIDAC notes that there is conclusive 


evidence to show that the establishment of a sur- 
veillance system is associated with reductions in 
infection rates. Surveillance is particularly useful 
in monitoring the effectiveness of infection-preven- 
tion-and-control programs. 

The Ministry requires long-term-care homes 

to have an ongoing program of surveillance to 
determine the presence of infections. As well, the 
standards set by Accreditation Canada, under 
which the three homes we visited were accredited, 
require homes to monitor infection rates. Recom- 
mendations regarding surveillance of infections 
that PIDAC has issued and that other organizations 
have published include that homes should: 

@ identify which infections they will track, 
based on evaluation of the types of infections 
for which their residents are most at risk. 
PIDAC suggests the homes consider track- 
ing various infections, including C. difficile, 
MRSA, VRE, FRIs such as influenza, skin infec- 
tions, and urinary tract infections. 

@ use standard definitions for determining when 
a resident has a particular infection and when 
infections should be counted in the home’s 
tracking system. This ensures that the infor- 
mation collected is consistent, accurate, and 
reproducible. 

@ regularly analyze the information gathered 
and identify trends signalling the need for 
corrective action, such as staff education or 
changes in practice. 

@ Have a certified Infection Prevention and Con- 
trol Professional (ICP) and/or trained individ- 
uals who have passed an education program 
endorsed by the Community and Hospital 
Infection Control Association—Canada. The 
ICP’s responsibilities generally include the 
surveillance of infections. 

The Ministry’s annual unannounced inspec- 

tion of each home identifies, among other things, 
whether homes have an ongoing program of surveil- 
lance in place to determine the presence of infec- 
tions. According to ministry information, over 95% 
of homes had such a program in place in 2008/09. 
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Selecting and Defining Infections 


None of the homes we visited had conducted a 
formal evaluation to identify which infections their 
residents were most at risk for. However, all the 
homes had informally selected certain infections to 
track and periodically added new infections based 
on circumstances in their homes. 

PIDAC has standard definitions for over 15 
infections. Most definitions are based on symptoms, 
such as “two or more loose watery stools within 24 
hours,” but some require laboratory confirmation. 
Two of the homes we visited had adopted all of 
PIDAC’s recommended infection definitions. The 
third home had adopted only three specific defin- 
itions and otherwise used general symptoms that 
may indicate an infection, such as cough, diarrhea, 
or vomiting. 


Tracking Infections 


At the homes we visited, resident-care staff, 
including personal support workers and nursing 
staff, identify if a resident has symptoms of an 
infection. Either the nursing staff or the home’s 
physician assesses the resident and if the resident 
has an infection, this information is recorded on 
an infection-control tracking form. Staff collect the 
forms monthly and input the data into each home’s 
electronic tracking system. Using their systems, the 
homes can compile infection statistics for review 
by various internal committees and/or senior 
management. 

We noted that all three homes used a manual 
form to track resident infections. Although they 
included certain infections, such as FRIs, skin infec- 
tions, and urinary tract infections, on their forms, 
many infections, including C. difficile and MRSA, 
were not specifically tracked. However, the homes 
informed us that they would record these infections 
in a category labelled “Other.” 

For their infection statistics to be as meaningful 
as possible and able to be compared with those of 
other homes, and to facilitate the most appropriate 
follow-up action, long-term-care homes should 


distinguish between infections acquired within the 
home and pre-existing infections of newly admit- 
ted residents. PIDAC’s best-practice document on 
surveillance (June 2008) states that any “health- 
care-associated” infection, such as ones acquired at 
long-term-care homes, is an infection that includes 
those occurring in the period beginning more than 
48 to 72 hours after admission of a resident. PIDAC 
has more specific guidance with respect to tracking 
C. difficile in its best-practice document (November 
2007), which indicates that the case is considered 
to be acquired in the home if the resident’s symp- 
toms occur more than 72 hours after his or her 
admission, or if the resident was readmitted with 
C. difficile that he or she had acquired at the home 
sometime in the previous four weeks. 

We found that the homes we visited had differ- 
ent policies for counting infections in their tracking 
systems. For example, one home used PIDAC’s def- 
inition to track C. difficile infections. Another home 
tracked C. difficile infections that occurred more 
than 72 hours after a resident’s admission, as well 
as all reoccurring infections, whether acquired at 
the home or elsewhere. The third home tracked all 
new and reoccurring C. difficile infections regard- 
less of whether they were acquired at the home or 
elsewhere. Because the three homes have different 
policies for tracking infections, their infection rates 
were not comparable. 


Reviewing and Reporting Infection Data 


We requested information for the 2008 calendar 
year on MRSA, VRE, C. difficile, FRIs, and urinary 
tract infections from each of the three homes we 
visited. Two of the homes reported no cases of 
either MRSA or VRE in 2008; the third reported 
only a few. With respect to C. difficile, although one 
home reported no cases, the other two reported six 
and 15 cases respectively. As well, we noted that 
the incidence of FRIs and urinary tract infections 
varied considerably among the three homes. With 
respect to FRIs, using data provided by the homes, 
we estimated that the number of cases per 10,000 
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resident days ranged from seven to 18. With respect 
to urinary tract infections, we estimated that the 
number of cases per 10,000 resident days ranged 
from four to 18. Each home established its own 
data-collection methodology, so the comparisons 
may not be exact. 

All three homes indicated their senior manage- 
ment review daily reports that highlight concerns 
about specific residents, such as the onset of new or 
worsening symptoms of an infection. 

PIDAC indicates that it is a best practice to evalu- 
ate infection rates against benchmarks. Bench- 
marks provide homes with a targeted maximum 
rate for infections and enable homes to evaluate 
their actual infection rate against the target. Pos- 
sible benchmarks that a home could use include the 
rate of infections in the home at a particular point 
in time in a prior year (known as a baseline rate), 
and the incidence rate of infections at other homes. 
A 2008 report issued by the Regional Infection Con- 
trol Networks noted that only 15% of non-acute- 
care facilities (primarily long-term-care homes) 
used external benchmarks, and 21% did not use 
any benchmarks at all. 

Although none of the three homes we visited 
had formally established baseline rates, all three 
indicated that they compare their current infection 
statistics against statistics from previous periods. 
For example, one home compared its infection 
statistics to infection statistics from prior months 
and from the previous year. Another home indi- 
cated that it generally does a month-to-month 
comparison of statistics four times a year. The third 
home said that it generally reviews each quarter’s 
infection data for changes from previous quarters. 
All three homes told us that their comparison of 
infection statistics in 2008 with statistics from prior 
periods did not identify any areas requiring cor- 
rective action. 

We also observed that two of the homes were 
comparing their number or rates of certain infec- 
tions against other homes. One of these homes had 
adopted a benchmark of having infections in its 
home be no more than the median of rates of infec- 


tion of the other homes against which it was track- 
ing. We noted that this home exceeded the median 
rate of certain infections during a number of 
months in 2008. For example, the home exceeded 
the median rate of FRIs in five of the six months it 
analyzed. The home told us that it communicated 
the results to staff and offered additional infection- 
control training. 

Under the Health Protection and Promotion Act, 
homes are required to report information to their 
local public health unit on certain diseases, such as 
tuberculosis and influenza, as well as outbreaks of 
respiratory infections and gastroenteritis (symp- 
toms of which include diarrhea and vomiting). The 
Ministry requires that homes report directly to it 
any outbreaks that the home reported to its local 
public health unit. However, homes do not have 
to report many other infectious diseases, such as 
MRSA and VRE. Hospitals have similar reporting 
requirements but, effective September 1, 2008, they 
must also identify when a gastroenteritis outbreak 
is caused by C. difficile. Long-term-care homes are 
not required to identify this. 

Ontario hospitals are required to report publicly 
on several patient-safety indicators including 
health-care-acquired infectious diseases, such as 
C. difficile, MRSA, and VRE, and on hand-hygiene 
compliance among health-care workers. Long-term- 
care homes, however, are not subject to similar 
reporting requirements. Although one of the homes 
we visited publicly posted information on the 
number of certain infections within the home, no 
other information on infection rates was publicly 
reported. As well, neither of the other two homes 
publicly reported any information on infection rates. 


Staff Training on Surveillance Activities 


PIDAC states that if resident-care staff, such as per- 
sonal support workers and nurses, are responsible 
for reporting infections or suspected infections, it is 
critical that they undertake training to ensure that 
potential infections are identified and reporting 
expectations met. As well, the Ministry requires 
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homes to provide an educational session to all staff 
annually that includes infection-control practices. 

A 2009 Canadian survey of infection prevention 
and control in long-term-care homes noted that 
almost one-third of responding homes identified 
as an important issue the need for infection- 
prevention-and-control education. In addition, 

a recent Ontario-wide survey, facilitated by the 
Regional Infection Control Networks, noted that 
47% of non-acute facilities—mostly long-term-care 
homes—indicated that educating staff on infection 
prevention and control was a key issue they were 
facing. 

None of the homes we visited had provided 
specific training to staff on their surveillance 
responsibilities, such as training on the case 
definitions for different types of infections and 
the reporting requirements for these infections. 
However, all three homes required new nurses and 
personal support workers to “shadow” more experi- 
enced individuals in the same position for varying 
lengths of time prior to working independently to 
learn their duties, which, we were informed, would 
include infection-surveillance responsibilities. As 
well, the orientation sessions for new staff at the 
three homes all included some discussion of sur- 
veillance. The homes also provided some specific 
training regarding certain infections, such as influ- 
enza, and indicated that the symptoms of the infec- 
tion are discussed to enable staff to better identify 
residents with infections. 

The average attendance rate, by topic, fora 
sample of educational sessions that we examined 
ranged from approximately 15% to 45% at the 
three homes. None of the homes tracked whether 
staff attended at least one educational session 
annually on infection control practices. Neverthe- 
less, we noted that, during 2008, two of the three 
homes held more than one session on each specific 
infection-control topic, and they would also pay 
staff to attend the session if they attended outside 
of their regular shift. However, the third home, 
which had the lowest attendance rates, only pro- 


vided one session per topic and did not compensate 
staff who attended. 


Infection Prevention and Control 
Professional 


According to PIDAC, the responsibilities of an Infec- 
tion Prevention and Control Professional (ICP) may 
include various functions, such as the surveillance 
of infections. The Ministry’s Long-Term Care Homes 
Program Manual requires that homes designate an 
ICP, and indicates that the individual should pos- 
sess expertise or be willing to acquire expertise in 
infection control. The Program Manual also states 
that the ICP is responsible for surveillance activities 
in the home. Further, PIDAC recommends that 
homes have a staffing ratio of one full-time ICP per 
150 to 200 beds. 

A recent Canadian survey of infection pre- 
vention and control in long-term-care facilities 
identified that only 8% of ICPs were certified and 
that there was only an average of 0.6 full-time- 
equivalent ICPs per 250 beds. This is well under 
PIDAC’s recommended ICP-to-bed ratio. In addi- 
tion, an Ontario survey of the non-acute-care sec- 
tor, primarily long-term-care homes, issued in 2008 
by the RICNs, identified that just 5% of ICPs were 
certified. 

Although each of the homes we visited had a 
person designated as its ICP, this role was on top of 
various other functions performed by these individ- 
uals. None of the designated ICPs had attended a 
program endorsed by the Community and Hospital 
Infection Control Association—Canada, nor had any 
of the ICPs obtained a Certification in Infection 
Control. The homes indicated that it has been a 
challenge to obtain and retain well-trained ICPs. 

None of the homes had an ICP-to-bed staffing 
ratio within the range recommended by PIDAC. 

All three homes told us that they did not have the 
resources to meet this recommendation. However, 
all three homes indicated that, if needed, they 
could contact other sources for information related 
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to infection prevention and control, such as their 
local public health unit or Regional Infection Con- 


trol Network. All three homes told us that, if neces- 


sary, their physicians could contact physicians at 
their local hospital who had expertise in infection 
prevention and control. 


RECOMMENDATION 4 


To enhance the effectiveness of infection- 
prevention-and-control programs, the Ministry 
of Health and Long-Term Care, in conjunction 
with the long-term-care homes, should: 


@ require long-term-care homes to identify and 


track infections in a consistent and compar- 
able manner, using standard definitions and 
surveillance methods; 

e establish reasonable targeted maximum 
rates/benchmarks for the more prevalent 
infections; and 

e look into requiring that long-term-care 
homes report publicly, as hospitals do, on 
certain patient-safety indicators, such as 
cases of C. difficile and hand-hygiene compli- 
ance among resident-care staff, using stan- 
dard definitions and surveillance methods. 


As well, long-term-care homes should ensure 


that staff, including designated infection- 
prevention-and-control professionals, have the 
infection-surveillance training recommended 
for their position. 


Current Ministry requirements on infection 
prevention and control outlined in the Program 
Manual state that every long-term-care home 
must have an ongoing program of surveillance 
to determine the presence of infections and 
provide education and training for all staff. As 
well, draft regulations under the new Long-Term 
Care Homes Act, 2007, include provisions on 
surveillance and education. In particular, the 
draft regulations will require homes to ensure 


that the presence of infections in residents is 
monitored and recorded, and that this informa- 
tion is analyzed daily and reviewed at least 
once a month to detect trends, for the purpose 
of reducing the incidence of infection and 
outbreaks. The computerized care-management 
system announced in January 2009 and 
expected to be fully implemented in homes by 
summer 2010 will help health professionals in 
homes assess and monitor the care needs of 
residents. In particular, this system will better 
enable homes to identify and assess residents 
with various infections, including MRSA, C. dif- 
ficile and respiratory infections. Further, the 
system will enable homes to track and monitor 
resident infections in a consistent manner using 
the same definitions, such that the data gath- 
ered will be comparable across all homes. The 
Ministry will receive data quarterly. 

The Ministry will review the appropriate- 
ness of establishing targeted maximum rates/ 
benchmarks for the more prevalent infections. 
However, the rates of certain infections, such 
as influenza and noroviruses (whose main 
symptoms include diarrhea and vomiting), in 
each long-term-care home often reflect the rates 
of these viruses in their local communities. As 
well, the rates are influenced by the vulnerabil- 
ity of the residents in the home. Therefore, the 
rates of these infections may not be reflective of 
the home’s internal infection-prevention-and- 
control practices. However, for other specific 
infections (such as skin infections), each home 
should establish its own baseline because this 
provides the home with the information neces- 
sary to assess the impact of the home’s infection- 
prevention-and-control program’s improvement 
activities over time. 

The Ministry fully supports public reporting 
and is looking into the reporting of patient 
safety indicators for long-term-care homes. 
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The homes generally supported this recom- 
mendation. However, one home was concerned 
with publicly reporting information on certain 
patient safety indicators because it believed it 
would be difficult to select the indicators and 
that there would be a negative public perception 
of homes unless the indicators were thoroughly 
explained. This home also highlighted that 
implementing best practices, reviewing whether 
practices are followed, and tracking indicators 
is very time consuming. Therefore, this home 
strongly believes the key to moving forward in 
this area is for the government to strengthen the 
overall staffing in long-term-care homes and, 
more specifically, to provide for a trained full- 
time Infection Control Practitioner (ICP) in each 
home. The home is also looking forward to the 
Provincial Infectious Diseases Advisory Commit- 
tee’s (PIDAC’s) release of additional information 
for long-term-care homes. 

Another home indicated that, while it con- 
tinues to identify infections using PIDAC’s case 
definitions, it has been working to improve the 
consistency of its tracking methods to ensure 
comparability of data. This home commented 
that clear definitions of outbreaks and guide- 
lines for tracking infectious illnesses would be 
helpful on a system-wide basis. The home noted 
that the establishment of targeted maximum 
rates/benchmarks that are applied to all long- 
term-care homes might be helpful in tracking 
system-wide success over time. With respect to 
public reporting, this home commented that 
additional research into meaningful meas- 
ures that reflect the unique circumstances of 
long-term-care home residents is needed. For 
example, the size of a given long-term-care 
home, the ability to restrict resident movement 
between areas within the home, and seasonal 
infection rates in the local population that visits 





the home are factors that influence infectious 
disease transmission. This home commented 
that, although various courses are offered to its 
staff, it agreed that a designated professional 
with infection-surveillance training would be 
of benefit to its residents. The home indicated 
a number of challenges in recruiting a nurse 

to take the ICP training program, such as the 
shortage of qualified nurses in its area, the need 
to maintain appropriate staffing levels, and the 
availability of funding. 

The third home commented that it will con- 
tinue to work with its Regional Infection Control 
Network and other infection control bodies to 
identify and obtain the most appropriate level 
and type of training and education for its ICP. 

As well, in the interim, the home will clarify the 
components of the ICP role and implement elec- 
tronic tracking of activities falling within each of 
these components to ensure complete coverage. 


Appendix—Details of PIDAC 





and Ministry Initiatives 


BEST-PRACTICE DOCUMENTS 


PIDAC has developed the following documents on 
best practices that are applicable to long-term-care 
homes. These documents incorporate the applic- 
able guidelines and recommendations from entities 
such as the Public Health Agency of Canada and the 
College of Physicians and Surgeons of Ontario, as 
well as recommendations from medical literature. 

e Best Practices for Cleaning, Disinfection and 
Sterilization (March 2006, revised April 
2006)—focuses on medical equipment. 

e Best Practices for Hand Hygiene (May 2008, 
revised January 2009)—includes guidance on 
when, why, and how staff in long-term-care 
homes should wash their hands. 
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@ Best Practices for Infection Prevention and 
Control Programs in Ontario (September 
2008)—includes guidance on the human 
resources and skills needed for an infection- 
prevention-and-control program, as well as 
the specific activities that should be included. 

e Best Practices for Infection Prevention and 
Control of Resistant Staphylococcus aureas and 
Enterococci (March 2007)—includes guidance 
on controlling the transmission of MRSA and 
VRE and managing residents with MRSA and 
VRE. 

e Best Practices Document for the Management of 
Clostridium difficile in all Health Care Settings 
(December 2004, most recently revised Janu- 
ary 2009)—includes guidance on identifying 
clusters of C. difficile, preventing their trans- 
mission, and managing residents with the 
infection. 

@ Best Practices for Surveillance of Health Care- 
Associated Infections in Patient and Resident 
Populations (June 2008)— includes guidance 
on tracking and monitoring health-care asso- 
ciated infections. 

e Preventing Febrile Respiratory Illnesses (Sep- 
tember 2005, revised August 2006)— includes 
guidance on detecting and containing clusters 
and outbreaks of common respiratory infec- 
tions, such as influenza. 

@ Routines Practices and Additional Precautions 
(August 2009)—includes guidance on 
reducing the risk of the transmission of micro- 
organisms. 

In addition, at the time of our audit, PIDAC was 
expecting to publish in spring 2010 a best-practice 
document that would provide additional guid- 
ance to long-term-care homes on environmental 
cleaning. 


CORE COMPETENCIES PROJECTS 


In response to the 2004 Final Report of the Ontario 
Expert Panel on SARS and Infectious Disease Control 


by Dr. David Walker and the Ministry’s Operation 
Health Protection plan, PIDAC and the Ministry 
developed educational material to enhance infec- 
tion-control training for front-line staff. In spring 
2007, the Ministry and PIDAC developed three 
educational modules: routine infection-control 
practices; hand hygiene; and the chain of infection 
transmission for hospital staff. These modules were 
posted on the Ministry’s website for health-care 
professionals. According to the Ministry, at the time 
of our audit, work was underway to adapt.these 
modules for other health-care workers and other 
facilities, such as long-term-care homes. The Min- 
istry also indicated that further educational mod- 
ules will be developed on topics such as additional 
precautions and surveillance. 


HAND HYGIENE IMPROVEMENT 
PROGRAM 


Proper hand hygiene (that is, using alcohol-based 
rub or soap and water to clean hands) by health- 
care workers is one of the most effective ways of 
preventing HAIs. In March 2006, the Ministry and 
the Public Health Agency of Canada held a work- 
shop to learn from the world’s leading authorities— 
such as the World Health Organization and experts 
from across Canada, the United States, and the 
United Kingdom—about programs that resulted in 
sustainable change in hand-hygiene practices. The 
workshop also discussed how these programs could 
be adapted for use in Ontario. On the basis of this 
workshop, the Ministry developed the “Just Clean 
Your Hands” Hand Hygiene Improvement Program, 
which it originally focused on implementing in hos- 
pitals. At the time of our audit, the Ministry indi- 
cated that it expects to have this program adapted 
for use in long-term-care homes by the end of 2009. 
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Chapter 3] Ministry of Education 
section 


Background 


The Ministry of Education (Ministry) is responsible 
for the system of publicly funded elementary 

and secondary school education in Ontario. Its 
responsibilities include developing the primary and 
secondary school curricula, setting requirements 
for student diplomas, and providing funding to 
school boards. The Ministry also set up the Educa- 
tion Quality and Accountability Office (EQAO)—a 
government agency—to provide independent 
assessments of student achievement by testing 
students in reading, writing, and mathematics. It 
is the responsibility of the Ministry and the school 
boards to review EQAO assessment reports and 
adjust strategy and training to foster continuous 
student improvement. The Ministry’s overall target 
is that 75% of all 12-year-olds achieve at least 

a level-three score, equivalent to a B grade, on 
province-wide EQAO testing for reading, writing, 
and mathematics. 

The Ministry’s Literacy and Numeracy Secre- 
tariat (Secretariat), the subject of this audit, was 
established in November 2004 to work with over 
4,000 elementary schools across 72 school boards. 
It serves English- and French-language schools in 
both public and Catholic school boards. The Sec- 
retariat’s mandate is to provide support to school 


“Miya Literacy and Numeracy 
Secretariat 


boards and schools to assist them in fulfilling their 
responsibilities to improve the achievement of chil- 
dren from junior kindergarten (JK) to grade 6, as 
measured by EQAO test results, and to close gaps in 
achievement for lower-performing student groups 
and schools. 

The Secretariat works to boost student achieve- 
ment by collaborating with school boards and 
schools to set targets, support boards in imple- 
menting secretariat initiatives, and foster a sense 
of goodwill and enthusiasm within the education 
sector. Along with funding a number of program 
initiatives designed to enhance teaching strategies 
and improve student achievement, the Secretariat 
employs over 80 experienced educators, called 
student achievement officers (SAOs), who work 
directly with schools and school boards across the 
province to implement strategies to improve read- 
ing, writing, and mathematics skills. 

Since it was established in 2004, the Secretariat 
has spent $340 million, with almost $288 million 
transferred to school boards to help them meet 
student-achievement targets. The details of these 
expenditures over the last five years are shown in 
Figure 1. 

On April 1, 2009, the two areas in the Ministry 
responsible for improving student achievement, the 
Literacy and Numeracy Secretariat (grades JK to 6) 
and the Student Success/Learning-to-18 Division 
(grades 7 to 12), came together to form the Student 


Literacy and Numeracy Secretariat 


Figure 1: Literacy and Numeracy Secretariat Expenditures, 2004/05-2008/09 ($ 000) 


Source of data: Ministry of Education 
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Schools on the Move 
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13,981 








Achievement Division. The responsibilities of the 
Secretariat will carry on substantially unchanged in 
the new division. The purpose of the amalgamation 
was to better align ministry responsibilities for all 
students from JK to grade 12. 


Audit Objective andScope — 


The objective of our audit was to assess whether the 





Literacy and Numeracy Secretariat had adequate 
procedures in place to: 

@ measure and report on the effectiveness of its 
activities in fulfilling its mandate to ensure 
that students in Ontario achieve a high level 
in reading, writing, and mathematics by age 
12; and 

@ ensure that its transfer payments to school 
boards are properly managed and directed 
to the areas in greatest need of support for 
students’ achievement levels to improve. 

The scope of our audit work included research- 

ing practices around student achievement in other 
jurisdictions; reviewing and analyzing ministry 
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files, administrative directives, policies, and 
procedures; and interviewing ministry staff as 
well as supervisory officers and principals at one 
French-language school board and five English- 
language school boards across the province. The 
school boards we visited were the Conseil scolaire 
de district catholique du centre-sud, Toronto Cath- 
olic District, Peel District, Thames Valley District, 
Lakehead District, and Thunder Bay Catholic 
District. Our audit also included a review of related 
activities of the Ministry’s Internal Audit Services 
Branch. We reviewed the Branch’s recent reports 
and considered its work and any relevant issues 
identified when planning our audit work. 


The Ontario government made a significant com- 
mitment to improving student achievement when, 
in 2004, it set a goal that 75% of all 12-year-olds 
(grade 6 students) would score a level-three stan- 
dard (approximately a B average) on province-wide 
testing for reading, writing, and mathematics by 
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2008. Although the Ministry of Education (Min- 
istry) had not achieved this goal by 2008, substan- 
tial progress has been made over the last five years, 
and the number of children achieving level three 
on Education Quality and Accountability Office 
(EQAO) testing has increased on average from 56% 
in 2003/04 to 65% in 2007/08 (this report uses 
2007/08 numbers because results for the 2008/09 
fiscal year were not available at the time of our 
audit). 

Further increasing the percentage of students 
achieving a B average on the EQAO literacy and 
numeracy tests will be a challenging undertaking. 
During our audit, we noted a number of areas that 
could be improved to help achieve this goal. 

Some of our more significant observations were: 

e Although the Literacy and Numeracy Secre- 
tariat (Secretariat) and the school boards we 
visited have done some limited assessments 
of how secretariat program initiatives have 
contributed to improving student achieve- 
ment, further analysis is required to assess 
the effectiveness of the various programs in 
improving student outcomes. Better analysis 
in this area would have enabled the Secre- 
tariat to ensure that its spending of almost 
$288 million was directed to the initiatives 
that provide the most benefit. 

e The information in reports submitted by 
school boards was insufficient to assure the 
Secretariat that funds were being spent on its 
initiatives and to track, over time, their impact 
on student outcomes. This lack of structured 
financial and performance reporting limits the 
Secretariat’s ability to carry out a comparative 
assessment of its program initiatives. Given 
that the Secretariat’s window of opportunity 
to help improve student achievement closes 
after students reach the age of 12, it is import- 
ant that the Secretariat develop effective 
improvement programs during those critical 
years. To be able to do so, the Secretariat must 
know which program initiatives work best and 
which should be modified or eliminated. 


e School board improvement plans were initi- 


ated to help teachers, principals, and school 
board staff plan and implement strategies to 
improve student achievement. The Ministry 
has developed a framework to help school 
boards and schools implement an effective 
improvement planning process. However, 
neither the Secretariat nor the boards we 
visited documented, monitored, or reported 
on the plans to the extent necessary to assess 
whether the plans were contributing to 
improved student achievement. Because it 
exercised only limited oversight, the Secre- 
tariat did not have the information needed 

to identify patterns and trends among school 
boards. Thus, it could not determine how 
effective the various plans were so that the 
most successful initiatives could be shared 
with other boards. 

Secretariat program funding was not always 
allocated to school boards and schools with 
the greatest need. Funding for some of the 
secretariat improvement initiatives was based 
on average daily enrolment (rather than on 
relative needs); in other cases, the Secretariat 
could not fully explain how the amount of 
funding that went to each school board was 
determined. For instance, we found that for 
one major program, the funding for the board 
with the greatest number of schools desig- 
nated as low-performing was only $17 per 
student, while several boards, with no schools 
designated as low-performing, received more 
than twice this amount per student. 

The Secretariat routinely uses certain boards 
as “bankers” to act as distributors of funds 

to third parties or other school boards. We 
question the need for such arrangements 

and noted that there is no Memorandum of 
Understanding or agreement between the 
Secretariat and the banker boards outlining 
respective roles and responsibilities, account- 
ability relationships, reporting requirements, 
and service levels to be provided. Also, the 


Secretariat paid banker boards administrative 
fees that in some cases appeared excessive. 

e Final report card marks for most students 
should be relatively comparable to their EQAO 
scores for reading, writing, and mathematics. 
The Ministry does not do this type of com- 
parison, but we carried out our own analysis 
comparing report card marks to EQAO scores. 
We noted that, at the school boards we vis- 
ited, approximately half of the student report 
card marks for grades 3 and 6 matched their 
EQAO scores. An additional 43% of the report 
card marks were within plus or minus one 
assessment level of the EQAO scores, with 
over twice as many of these report card marks 
exceeding EQAO results versus falling short of 
them. We believe this type of analysis would 
be useful to conduct—significant differences 
could highlight areas where adjustments to 
student assessment practices or EQAO testing 
warrant consideration. 


We would like to thank the Office of the Auditor 
General for the work put forth in preparing this 
report; the recommendations will be of assist- 
ance to the Literacy and Numeracy Secretariat 
as it refines its future work. 

In 2008, a leading research company evalu- 
ated initiatives undertaken by the Ministry to 
raise student achievement between February 
2007 and October 2008 and concluded “that 
over its brief history, Ontario’s Literacy and 
Numeracy Secretariat has had a major and 
primarily positive impact on Ontario’s education 
system.” In six years, student achievement has 
increased from 56% to 67% of students reach- 
ing level 3 in the grade 3 and 6 EQAO tests, 
the achievement gaps for English language 
learners and students with special needs has 
been reduced, and the percentage of Ontario 
elementary schools in which fewer than 50% 
of students were meeting provincial expecta- 
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tions has been reduced from 19% to less than 
5%. Ontario’s high school graduation rate has 
increased from 68% to 77% of students gradu- 
ating within five years. Ontario is recognized 
internationally as a school system of “excellence 
and equity,” a leader in achieving multi-year 
continuous improvement in student outcomes, 
both increasing student achievement and 
increasing equity for diverse student groups. 

The Ministry has provided leadership in 
creating a significant shift in the teaching and 
learning culture of Ontario’s schools. This has 
required that the Secretariat function in a con- 
tinuous cycle of research, dialogue with school 
boards and schools, development of strategies, 
pilot implementation, assessment, review, 
refinement, and expansion. 

Ministry initiatives work in an integrated 
Way to improve student learning and achieve- 
ment and narrow the gaps. While individual 
programs are evaluated to varying degrees, it is 
difficult to quantify the degree of impact of indi- 
vidual strategies. Evidence that our approach 
has credence exists in the analysis of outcomes, 
which reaffirms that the Ministry, in partnership 
with district school boards, schools, and staff, 
is significantly improving student achievement 
and closing the gap. The Ministry recognizes 
that some areas need to be strengthened, and 
the recommendations will help us to do so. 





Detailed Audit Observations 


MEASURING AND REPORTING ON 
PROGRAM EFFECTIVENESS 


The Ministry’s priorities are to attain high levels 
of student achievement, close gaps in student 
achievement, and maintain high levels of public 
confidence. It has implemented a number of initia- 
tives to ensure that more students succeed. One 
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of these initiatives is the Literacy and Numeracy 
Strategy, which identifies practices that improve 
student achievement. The government’s Reach 
Every Student strategy affirms that strong literacy 
and numeracy skills are the critical foundation 

for all other academic achievement. Accordingly, 
the Secretariat’s goal is to ensure that all students 
achieve a high level of literacy, numeracy, and 
comprehension, with a province-wide total of 75% 
of 12-year-olds (grade 6) meeting the provincial 
standard (level three or a B average) on reading, 
writing, and mathematics testing by 2008. Figure 2 
shows EQAO results over the last five years. 

Over the last five years, the average results for 
both English- and French-language schools have 
gone up overall by almost 10%, from 56% to 65%. 
Although French-language schools achieved the 
Ministry’s goal in 2007/08, with at least 75% of 
12-year-olds scoring a level three in reading, writ- 
ing, and mathematics, English-language schools 
have not. The percentage of English-speaking 
12-year-old students at the provincial level-three 
standard is still only 65%. However, the percent- 
ages of students who scored at level 2.7 (or about 
10% below the target level three) in reading, 
writing, and arithmetic were 79%, 85%, and 75%, 
respectively. This suggests to us that the Ministry’s 
goal to have 75% of 12-year-olds achieve at least 


a level-three score on EQAO testing, while chal- 
lenging, is not unreasonable. The Ministry recently 
reiterated this goal. 

At a symposium sponsored by the Secretariat in 
November 2005, the Premier stated that four out of 
every 10 students were not meeting the provincial 
standard. He further stated, “Those children are at 
risk. They are at risk of doing poorly throughout 
their school years. They are at risk of dropping 
out of high school. They are at risk of growing up 
and achieving less than they are capable of.” Bet- 
ter test scores are a sign of progress, but from the 
results discussed above, it is evident that in 2008 
there were still 3.5 out of every 10 students who 
were not meeting the provincial standard, and who 
could therefore be at risk. In this regard, although 
progress is being made, the Ministry has not yet 
met its overall goal of 75% of 12-year-olds achiev- 
ing at least a level three on EQAO testing by 2008. 
It advised us that it plans to achieve the goal by 
continuing with a number of initiatives, including: 

e strengthening the networks of educators who 
focus on student work and effective pedagogy 
so they can learn from and with each other; 

@ supporting the work of the School Effective- 
ness leads through regional professional 
learning sessions; 

e building capacity at the board level; 


Figure 2: Percentage of Grade 6 Students at or Above the Provincial Standard (EQAO Level 3 or a B Average) | 


Source of data: Education Quality and Accountability Office 
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@ supporting board improvement planning ; 

e building on and sustaining the School Effect- 

iveness Framework; 

@ continuing field-based support through the 

student achievement officers; 

e intensifying support for low-performing 

boards; and 

e analyzing and using data to target improve- 

ment strategies to the areas of greatest need. 

The Ministry relies on EQAO test results to 
measure the success of its programs. However, this 
approach to measuring success does not consider 
the effect of programs on the same students over 
time. Other jurisdictions track the test scores of 
individual students over time, for example, by 
comparing a particular group of students’ grade 3 
results with their grade 6 results three years later 
(this is called “cohort tracking”). In our view, if the 
Ministry did this type of tracking in addition to the 
test-score analysis it currently undertakes, it would 
have a better measure of the value its investment in 
programs and initiatives has achieved. 

Another measure of student achievement relates 
to the range in scores between high-performing 
and low-performing student groups and schools. 
Educational research shows that jurisdictions that 
have narrower gaps in achievement between the 
top- and bottom-performing student groups and 
schools also tend to have higher average scores than 
do those where the gap is larger. Yet Ontario does 
not publicly report on how wide the gap is between 
the highest and lowest performers. This measure- 
ment would help the Ministry evaluate whether it 
is meeting its goals and responsibilities for student 
achievement, and enable it to track, for selected 
student groups and schools, whether the gap is 
being reduced over time. 


RECOMMENDATION 1 


The Ministry of Education should develop 
more comprehensive indicators for measuring 
and reporting on its effectiveness in improving 
student achievement. In addition to reporting 
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the percentage of 12-year-olds who are at or 
above the provincial standard, it should also 
consider reporting changes in the gap between 
the top-performing and lower-performing stu- 
dent groups and schools, as well as how specific 
student cohorts perform over time while partici- 
pating in the programs and initiatives intended 
to improve their performance. 


The Ministry agrees that additional indicators 
would be useful in reporting its effectiveness in 
improving student achievement. In September 
2009, the Education Quality and Accountability 
Office (EQAO) began reporting information on 
tracking student cohorts and how they perform 
over time. This analysis was dependent on the 
Ministry’s full implementation of the Ontario 
Education Number to track each student. 

In addition, the EQAO produces for the 
Ministry and school boards a report of the test 
results for specific student groups. The Ministry 
uses this information to track over time the gap 
in achievement between the overall student 
population and specific student groups (for 
example, English-language learners, students 
with special needs, and boys). Also, the Ministry 
tracks trends over time in the proportions of stu- 
dents at each of levels 1 through 4. This infor- 
mation informs ministry planning and priorities 
from year to year. 

The Ministry will continue to examine 
and use this information to further assess the 
effectiveness of its programs and to refine its 
initiatives in improving student achievement 
and narrowing the gaps. 


SCHOOL BOARD IMPROVEMENT PLANS 


School board improvement plans are intended to 


help teachers, principals, and senior school board 


staff plan and implement strategies to improve 
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student achievement in both the short and long 
terms. Improvement plans are also a mechanism 
through which the public can hold schools and 
boards accountable for student success and through 
which improvement can be measured. An effective 
improvement plan includes strategies for improve- 
ment, indicators of success, performance targets, 
relevant timelines, and reporting on student 
achievement. 


Completion and Review of Improvement 
Plans 


School boards, superintendents of education, 
principals, and teachers all have varying roles in 
preparing and monitoring improvement plans at 
the school board and school levels. The Ministry 
has produced a number of guidelines to help school 
boards prepare effective improvement plans. One 
of these guidelines is the School Effectiveness 
Framework, which provides key indicators of school 
effectiveness for the purpose of building board and 
school capacity to: 

e identify strengths and weaknesses; 
perform self-assessment and analysis; 
achieve better improvement planning; 
implement high-yield strategies; and 


determine the monitoring and feedback 
strategies necessary for improvement and 
accountability. 

In early 2008, a consultant the Ministry engaged 
to review school board improvement plans reported 
that there was a wide variety of plan formats, 
which made it a challenge for the Secretariat to 
identify patterns and trends from board to board. 
In addition, the absence of a common planning 
template means less usage of a common language 
that could help school boards to duplicate the suc- 
cessful practices of other boards. In response to this 
report, the Ministry adopted “SMART” goals—that 
is, goals that are specific and strategic, measurable, 
achievable, result-based, and time-bound—to be 
incorporated into improvement plans. The Ministry 


recommended that school boards develop their 
improvement plans based on these SMART goals. 
Although there is no formal requirement that 
improvement plans be submitted to the Ministry, 
we found that all 72 school boards have been sub- 
mitting their plans every year. We noted, however, 
that once the Secretariat received the plans, there 
was limited documentation to demonstrate that 
the plans were reviewed to ensure that all the 
required components of an effective plan had been 
addressed. We selected a sample of 13 improvement 
plans of large and small school boards and noted 
that some school boards set SMART goals that 
were vague, difficult to measure, and not within 
the board’s direct control. To realize the benefit of 
its SMART initiative, the Ministry should review 
improvement plans and document each board’s 
degree of consistency with the SMART guidelines. 
Among the responsibilities of the Ministry’s 
student achievement officers is working with school 
boards to set student achievement targets and 
develop improvement plans. However, there was 
no documentation of feedback to the school boards 
on whether they had included all the required 
components of an effective plan. We were informed 
that student achievement officers provide verbal 
feedback only. Without consistent and concise 
documentation of these informal discussions, there 
is no assurance that any possible shortcomings in 
the improvement-planning process have been com- 
municated to the proper levels of authority within 
the school board. This lack of documentation, com- 
bined with the lack of detailed consistent reviews, 
hinders the Secretariat in its ability to follow up 
on whether any previously observed shortcomings 
have been addressed. 


Monitoring and Reporting the Achievement 
of Plans 


In our review of the improvement-planning process, 
we noted that the accountability mechanisms in 
place at the Ministry were informal, with little or 
no documentation. The Secretariat does not require 


that school boards periodically report the results of 
the monitoring and measurement of their improve- 
ment goals. Rather, it is left up to school boards to 
decide when and how to monitor the improvement 
plans. As a result, the Ministry has little assurance 
that school boards are meeting the goals or time- 
lines in their improvement plans and that, where 
necessary, corrective action is being taken to meet 
goals. We noted that British Columbia uses a formal 
achievement contract with school boards to more 
closely monitor their improvement plans. This 
contract requires the boards to submit the goals, 
objectives, performance indicators, and progress 
made to improve student achievement. In addition, 
any goal not met is either to be carried forward 
until it is achieved or deleted with an explanation 
of why it is no longer relevant. 

We reviewed the improvement plans for 13 
school boards and noted that 10 of them did not 
provide information on the progress made in the 
implementation of past improvement strategies. All 
13 of the Ontario boards in our sample had mech- 
anisms in place to track and report on the achieve- 
ment of goals, but this was generally not done. We 
noted that in Alberta the Ministry of Education 
provides funding for student improvement projects; 
school boards are required to submit annual reports 
that include financial information as well as results 
on student learning outcomes achieved, lessons 
learned, and effective practices that demonstrate 
the greatest impact on student learning. Without 
proper documentation, monitoring, and objective 
evaluations of the results of improvement strat- 
egies, it is difficult for the Secretariat to ensure 
that school board improvement plans are achieving 
their intended goals. More detailed board improve- 
ment plans that assess whether their goals have 
been achieved, with recommended actions when 
goals are not met, would assist the Secretariat in 
its review of the results of school board program 
strategies. 

To help it identify overall provincial trends, the 
Secretariat receives an annual mid-term review 
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from the 72 school boards. These mid-term reviews 
include responses to a number of inquiries from 
the Secretariat, including a summary of the board’s 
initiatives to improve student achievement. This 
report is used mainly as a planning document 

by the Ministry and also as a school board self- 
assessment tool. What the report does not include, 
however—and what would make the report more 
useful for the Ministry as it tries to determine 
trends—is information on the achievement of 
improvement-plan goals. 

If the Secretariat determines that a school board 
has performed poorly in any of the areas reported 
on in a mid-term review, there are no formal pro- 
cedures in place to take corrective action. We were 
informed by ministry staff that, instead, informal 
discussions are held with the board in partnership 
with the Ministry’s student achievement officers to 
develop strategies for improvement. We found no 
documentation with respect to this process. In addi- 
tion, none of the boards we visited had received 
any report-back from the Secretariat indicating 
compliance with ministry expectations for board 
improvement or corrective actions that might be 
necessary. Staff at one school board we visited said 
that it would be beneficial if the Secretariat would 
provide written feedback on the mid-term reports 
to avoid any misunderstanding, which sometimes 
occurs when student achievement officers provide 
only verbal feedback. Formal feedback would also 
provide the Ministry with a more objective basis for 
assessing the effectiveness of student achievement 
officers. 

Furthermore, we noted that school improve- 
ment plans are generally not made available for 
parents and other stakeholders to review. Of the 13 
school boards we reviewed, only three had posted 
their improvement plans on their board’s website. 
In British Columbia, the school board achievement 
contracts, along with the improvement plans from 
every school district, are available on board and 
school websites for review by interested parties. 


Chapter 3 ¢ VFM Section 3.07 





194 4 2009 Annual Report of the Office of the Auditor General of Ontario 


~~ 
°S 
ise] 
(=! 
° 
= 
oO 
o 
77) 
= 
Le 
=> 
e 
ise] 
= 
o 
— 
i-5 
cS 
= 
(os) 





RECOMMENDATION 2 


To ensure that the improvement-planning pro- 
cess is sufficient to support boards, administra- 
tors, principals, and teachers in helping students 
to improve results and progress toward the 
provincial standard in achievement testing, the 
Ministry of Education should: 

e@ implement a formal improvement-plan 
review process to help ensure that all of the 
necessary components of an effective plan 
are included; 

@ require that school boards post improvement 
plans online to enhance accountability and 
transparency; 

e consider adopting the practice followed 
in some other provinces of using a formal 
contract with school boards that would 
require school boards to periodically report 
their results in achieving the goals in their 
improvement plans; and 

e properly document the result of its monitor- 
ing efforts along with any required correct- 
ive action to be taken and any subsequent 
follow-up where plans are not complete. 





The Ministry agrees that the board-improve- 


ment-planning process and documentation 
should be improved. A board improvement 
plan is designed by boards to articulate the 
actions required to improve student learning 
and achievement K-12. This is most effectively 
accomplished through a combination of pres- 
sure and support. 

Since 2005/06, the Ministry has provided 
annual feedback to school boards to support the 
ongoing refinement of their improvement plans. 
In 2007, the School Effectiveness Framework 
was created to guide school and board analysis 
and improvement planning. In 2008, a global 
leader in enhancing student achievement and 
improvement planning was commissioned to 


perform a comprehensive review of all board 
improvement plans. It made recommendations 
through a written report back to each board 
on ways to improve plans and provided the 
Ministry with a detailed analysis, which formed 
the basis for refinement in the current year. In 
2008/09, the Ministry developed a template 
of expected components for an effective plan. 
School board plans were gathered in June 2009 
and a written review has been provided to 
each board identifying areas for improvement. 
Revised plans, which also incorporate an analy- 
sis of the most recent Education Quality and 
Accountability Office (EQAO) data, are to be 
submitted to the Ministry in October. These will 
form the basis for three meetings between min- 
istry staff and school board staff throughout the 
year as they review plans, progress made, and 
expected outcomes relating to student learning 
and achievement. . 
The Ministry will continue to work with 
school boards to enhance the improvement 
planning process, including more effectively 
documenting feedback to and discussions with 
boards regarding their plans and actions. The 
Secretariat will also encourage boards to post 
their board improvement plans online. 


MONITORING AND FUNDING OF 
PROGRAM INITIATIVES 


To achieve its goals of attaining high levels of 
student achievement and closing gaps in that 
achievement, the Secretariat and the Ministry have 
implemented a number of initiatives to ensure that 
more students succeed. The programs currently 
offered by the Secretariat are as follows: 

@ Ontario Focused Intervention Partnership 
(OFIP)—Schools identified as low-per- 
forming or static are given targeted support, 
such as allowing teachers time away from 
the classroom te-participate in professional 


development activities that will help them to 
increase their effectiveness. 

@ School Effectiveness Framework—To improve 
the way planning is done, this program helps 
boards and schools to identify areas that 
require attention and implement strategies 
for enhancing strengths and addressing 
weaknesses. 

@ Schools Helping Schools—High-performing 
schools work with low-performing schools 
to help the latter improve student perform- 
ance. The program also provides networks for 
schools and boards to learn from each other. 

@ Capacity Building—Strategies are developed 
to help improve practices throughout the 
school system. For example, investments 
are made in initiatives that enhance teacher 
knowledge and skills. 

e Tutors in the Classroom—College and univer- 
sity students tutor elementary school students 
in literacy and mathematics under the super- 
vision of classroom teachers. 

e@ Schools on the Move—Schools that have 
made substantial progress in raising student 
achievement and sustained this progress over 
several years are highlighted to share success- 
ful practices with other schools. 


Monitoring Student Achievement Initiatives 


To develop its student achievement initiatives, the 
Secretariat gathers information about the status of 
student achievement in the province and how to 
best support large-scale change. It looks at lessons 
learned from international educational reform 
efforts, such as research on how to effectively 

build and sustain improvement. The programs the 
Secretariat develops are also based on advice from 
educational advisors hired by the Ministry. Overall, 
the main impetus behind program initiatives is the 
Secretariat’s collaboration with school boards to set 
targets, support board-identified projects, and build 
capacity, all with the goal of improving student 
achievement. 
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Program delivery happens at the board level. 
School boards are responsible for implementing 
the Secretariat’s program initiatives for improved 
student achievement. But the Ministry is ultimately 
responsible for achieving its goal of having 75% of 
12-year-olds (grade 6 students) meet the provincial 
standard on province-wide reading, writing, and 
mathematics testing. According to educational 
research, the level of educational accomplishment 
reached by age 12 is generally seen to set the pat- 
tern for future learning and academic success that 
will help students to develop adequate skills to 
pursue lifelong learning and expand their career 
opportunities. At a 2005 symposium sponsored 
by the Secretariat, a former Minister of Education 
added to this idea, noting that educators must 
“understand and deal with as many of the child’s 
challenges by age eight and have the job essentially 
done by age twelve.” In other words, there is a rela- 
tively small window of opportunity to help improve 
student achievement. Therefore, it is important that 
the Secretariat’s interventions be timely and that it 
develop—and ensure that school boards deliver— 
effective improvement programs to increase 
student achievement during those crucial years. To 
be able to do so, the Secretariat must know which 
program initiatives work best and which should be 
modified or eliminated. 

Government guidelines for transfer-payment 
accountability require that ministries have the 
oversight capacity to ensure that recipients (in this 
case, the school boards) are using the funds for 
the intended purpose and achieving the desired 
results. Appropriate oversight would include com- 
munication with the school boards on a regular 
basis, ongoing monitoring to ensure that objectives 
are achieved (including receiving reports from 
school boards), and taking corrective action when 
necessary. The school boards we visited had not 
carried out sufficient assessments of secretariat 
initiatives and how these contributed toward 
improving student achievement. Although the 
Ontario Focused Intervention Partnership (OFIP) 
program is assessed annually, a more complete 
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assessment of its other initiatives would help the 
Secretariat demonstrate the effectiveness of all 

its programs. As a result, we questioned whether 
the Secretariat or school boards have adequate 
information to know whether secretariat resources 
are being directed to the program initiatives that 
provide the most benefit. 

The Ministry informed us that it was difficult 
to evaluate each program individually, that it is 
the sum effect of all its programs that contributes 
to student achievement, and that a number of 
strategies have been pursued to support learning. 
The Ministry informed us that, for instance, it 
has achieved two consecutive four-year collective 
agreements between school boards and their staff. 
As well, and in contrast to the $288 million the Sec- 
retariat has spent on its programs over the last five 
years, $1.4 billion has been spent on the Primary 
Class Size Reduction program over that time per- 
iod. The Ministry advised us that during 2008/09, 
the original goal for this program was achieved, 
with 90% of kindergarten to grade 3 classes having 
20 or fewer students and all primary classes having 
23 or fewer students. 

While acknowledging the Ministry’s position, we 
still believe that it would be useful to identify which 
programs and initiatives work—and do not work as 
well—in schools. Surveys, focus groups, and even 
anecdotal evidence may be useful in determin- 
ing the effectiveness of initiatives, highlighting 
best practices, and disseminating these practices 
throughout the province’s school boards. 


Program Funding 


Funding for the Secretariat’s programs is allocated 
to school boards. We reviewed how funding was 
allocated for six secretariat programs and found 
that funding for Tutors in the Classroom was allo- 
cated on the basis of need and that a set amount 
was provided for each school for Schools on the 
Move. However, over $200 million had been spent 
on the other four programs over the last five years, 
and either the funding was based on average daily 


student enrolment or the Secretariat could not fully 
explain the method it used to allocate funding. 

Funding that is based on average daily enrol- 
ment, as opposed to relative need, does not focus 
scarce resources on the highest priorities that have 
been identified by assessments such as EQAO test- 
ing, nor does it target funding to low-performing 
schools or boards. This method of funding also 
does not sufficiently account for the fact that some 
schools have either students who are more challen- 
ging to educate than others or just a much higher 
percentage of students who need additional help. 

One of the programs for which the Secretariat 
could not fully explain its funding method was the 
Ontario Focused Intervention Partnership. Our 
review indicated that funds were allocated partly 
on the basis of enrolment and partly on the basis 
of schools’ needs, but the Secretariat could not 
explain the rationale for much of the program’s 
funding. Funding for school boards with 1,000 stu- 
dents or more ranged from $13 to $83 per student. 
The board with the greatest number of schools 
designated as low-performing (75 schools where 
less than half the students achieved the provincial 
standard) received funding of only $17 per student 
while several boards with no schools designated 
as low-performing received twice this amount per 
student. Such inequities are partly due to the fact 
that a major portion of Ontario Focused Interven- 
tion Partnership funding is based on prior years’ 
allocations, with annual additions for new funding. 
Under such circumstances, inequities can develop 
and become more evident over time. 

Government accountability guidelines for trans- 
fer payments stipulate that agreements for funding 
should include specific, measurable results for the 
initiative being funded and should also indicate 
reporting requirements. School boards should there- 
fore be required to report on the use of funds and 
the results achieved. We noted that not all school 
boards submitted reports to the Secretariat and that 
there was little follow-up to obtain these reports. 
From our review of the reports that were submitted 
by school boards, we noted that the financial and 


outcome information they provided was insufficient 
for the Secretariat to know whether funds were 
spent for their intended purpose, whether there 
were any unspent funds, or whether the outcomes 
of the funding were achieved. During the 2008/09 
fiscal year, the Secretariat began to recover unspent 
funds by reducing the current year’s grants, but 

this was possible for only those school boards that 
reported spending by program initiative. 


Use of Financial Agents 


In some cases, the Secretariat uses financial agents 
to act as distributors of ministry funds to third par- 
ties or other school boards. These agents include 
the Council of Directors of Education (CODE) and 
a number of school boards, and are collectively 
referred to as “banker boards.” Our review of these 
financial arrangements generally indicated that 
proper accountability measures to effectively mon- 
itor the banker boards and ensure that government 
funds were being spent appropriately were not in 
place. Specifically, our concerns included: 

@ Since the 2004/05 fiscal year, the Secretariat 
has paid banker boards $1.1 million in admin- 
istrative fees. We calculated the amount of the 

‘fees paid compared to the amount of the con- 
tracts and found that, although administra- 
tion fees averaged about 13%, some fees were 
substantially more and appeared excessive. 
We also questioned the need for administra- 
tive fees if the Secretariat instead made all its 
payments directly to school boards. 

@ Since the 2004/05 fiscal year, often near 
year-end, the Secretariat has advanced funds 
to banker boards without specifying their pur- 
pose or its expectation that the funds be trans- 
ferred to recipient boards on a timely basis. 
As a result, some of the funds have remained 
unspent over a number of years. Figure 3 
shows a list of unspent funds at CODE and at 
the school board that received the greatest 
amount of advanced funding. 
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e@ We noted the contracts between the Secretar- 
iat and CODE and the banker boards that we 
reviewed were vague. They did not contain 
specific requirements such as the purpose 
of the funds advanced, the specific measur- 
able deliverables, and the time frame of the 
contract. Without a properly constituted 
agreement, it is difficult for the Secretariat to 
ensure that funding expectations and obliga- 
tions are fulfilled. 

@ Because the Secretariat had poor financial 
records and controls, it could not provide 
us with a list of the total funds on deposit 
with various banker boards. However, we 
were able to determine that one board was 
the Secretariat’s main distributor of funds to 
the other boards. Our review of the largest 
banker board showed that, of the $22 million 
advanced to the board, the Secretariat had 
information on how only $5.8 million had 
been spent. The Secretariat could not provide 
us with documentation showing that it had 
approved payments or where the remaining 
balance had been spent. We contacted the 
board and were provided with a list of the 
fund recipients, but we could not determine 
whether the funds disbursed were used for the 
intended purpose. 

e@ The Secretariat did not properly monitor the 
use of funds at CODE. Because the Secretariat 
either did not maintain proper financial 
records or had only limited documentation, 
we selected a sample of 16 payments CODE 


Figure 3: Funds Unspent by Two Banker Boards 


Source of data: Council of Directors of Education and York Region District 
School Board 





Funds Funds Unspent at 
Advanced Year End (Cumulative) 
8,343,699 8,090,537 
26,400,000 


Fiscal Year 
2004/05 















2005/06 —- 26,133,689 
2006/07 15,500,000 =  —-229,060,115 
2007/08 4,927,149 18,126,319 
2008/09 1,350,000 —« 12,126,811 
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had made to determine whether the expendi- 
tures were supported by documentation. 
We obtained this information directly from 


CODE and found that of the 16 disbursements 


(totalling $555,000), only two of them were 


made under a letter of agreement, contract, or 


proposal, and only four payments were sup- 


ported by invoices. We were not able to match 
$108,000 of the total $555,000 in payments to 


specific funding proposals. Furthermore, we 
reviewed expenditure documentation for 86 


proposals that had been approved for funding, 


and found that the Secretariat could provide 
information to support only $18 million in 


expenditures out of a total of $24 million paid 


out by CODE. 
The Secretariat needs to implement proper 


The Ministry agrees with the recommendation. 
To assess the effectiveness of the Ministry’s 


' overall strategy to improve student achieve- 


ment, an external review by a leading Canadian 
research company was commissioned. The 
report noted that “over its brief history, the Sec- 
retariat has had a major and primarily positive 
impact on Ontario’s education system.” In addi- 
tion, the most recent annual evaluation of the 
Ontario Focused Intervention Program (OFIP) 
demonstrates that OFIP schools achieved 
improvements in student achievement that were 
at least double the improvements observed in 
schools in general. A current analysis of the 
impact of the School Effectiveness Review pro- 
gram indicates that schools involved in district 


financial controls, maintain complete financial 
records, and put better monitoring procedures into effectiveness reviews achieved greater gains 
place. As well, the Secretariat should reconsider the than did schools in general. The Ministry will 


need to pre-flow funds to banker school boards. develop a schedule to review its current pro- 
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RECOMMENDATION 3 





To ensure that student achievement initiatives 
are effective and that limited resources are used 
appropriately, the Literacy and Numeracy Secre- 
tariat should: 

@ formally evaluate how well all its program 
initiatives contribute to improving student 
achievement, and modify or eliminate the 
less effective initiatives; 

@ ensure that its program funds are allocated 
to the areas of greatest need; 

@ ensure that program funds are being spent 
for the intended purpose; 

@ ensure that expenditures made by the Coun- 
cil of Directors of Education are appropri- 
ately approved and supported; and 

@ reconsider pre-flowing funds to “banker” 
school boards. 


grams over a three-year period. 

Funding decisions within specific programs 
are made in accordance with several factors: 
enrollment, number of low-performing schools, 
geographic and distance factors, as well as the 
readiness of a board or school to implement the 
changes needed. For example, one northern 
board covering a large geographic area, with 
1,357 students, received $112,585 in funding, or 
$83 per student, while a board in the south with 
more than 240,000 students received more than 
$4 million, or $17 per student. The Ministry 
accepts that it needs to be more transparent and 
document its funding decisions for each of its 
program areas. 

The Ministry also acknowledges that 
improved monitoring of the use of funding by 
boards would better inform decision-making. 
During the 2008/09 school year, boards were 
required to report spending by program and 
funding allocation and provide feedback on 
the implementation and impact of the various 


w 


initiatives. The Ministry will review and improve 
this process and its documentation. Since the 
audit, the Ministry has received documenta- 
tion that supports the full amount paid out 

by the Council of Directors of Education. The 
Ministry will assess its use of lead boards as well 
as review and improve its financial oversight 
procedures and documentation with respect to 
its financial agents to be in accordance with the 
government’s Transfer Payment Accountability 
Directive. 


CONSISTENCY OF STUDENT 
ASSESSMENTS 


In the elementary education system, a number 

of student assessments are made throughout the 
school year. Of all these assessments, the only ones 
that parents generally see are report card marks 
and the EQAO test scores for grades 3 and 6. It is 
important that the information provided by these 
assessments is relatively consistent. In our 2003 
audit of Curriculum Development and Imple- 
mentation, we noted that one possible method of 
measuring consistency in student assessment for 
some subjects is to compare report card marks to 
EQAO scores. We recommended that the Ministry 
implement procedures to monitor and report on 
consistency in the student assessment practices 
used in the province. Comparing report card marks 
to the EQAO scores could highlight areas that the 
Ministry or EQAO may need to address, identify 
students who need to be followed up on, and assist 
in planning. A more in-depth review may reveal 
individual schools, classes, or students that may be 
outliers. 

During our current audit, we noted that the Min- 
istry did not have procedures in place to assess the 
consistency between report card marks and EQAO 
scores. From our visits to school boards, we noted 
that some boards do compare the marks with the 
scores. However, such comparison is mainly left up 
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to school principals and teachers to do when plan- 
ning and developing strategies for helping students 
improve overall achievement. 

At each school board we visited, we did our own 
comparison of the report card marks—for reading, 
writing, and mathematics—to EQAO scores (of 
an average of 22,500 students in grades 3 and 6). 
Figure 4 shows our analysis of the last two years for 
both grades. 

Approximately half of the student report card 
marks for both years matched EQAO scores. An 
additional 43% of report card marks were within 
plus or minus one assessment level of the EQAO 
scores. We also noted that in both years over twice 
as Many report card marks exceeded EQAO results 
than fell short of them—31% of report card marks 
were higher than the EQAO result while 15% of 
report card marks were lower. We believe that the 
Ministry should conduct a similar comparative 
analysis and ensure that school boards follow up 
on significant differences. For instance, the fact 
that the EQAO score of 4% of students bears no 
resemblance to their report card mark could indi- 
cate issues requiring teacher training on curriculum 
expectations and what student work at each 
achievement level should look like. 





Figure 4: EQAO Scores Compared to Report Card 
Marks (%) for the Schools We Visited 


Source of data: Education Quality and Accountability Office, and Ministry of 
Education 
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RECOMMENDATION 4 


To help ensure that students are being assessed 
in a consistent way, the Ministry of Education 
should monitor the results from different types 
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of assessment, especially those from report 
cards and Education Quality and Accountability 
Office (EQAO) tests, to identify any major dis- 
crepancies for follow-up. 





The Ministry agrees with this recommendation. 
Over the past three years, the Ministry has 
worked with school boards to develop the tech- 


nical capacity to gather student and school data 
to permit analysis. The Ministry now holds stu- 
dent report card data over several years and is in 
a position to undertake a study of student report 
card and EQAO results to determine the degree 
of comparability between the two indicators and 
to identify appropriate areas for follow-up. In 
early October, the Ministry issued a competitive 
tender to initiate this study. 

At present, in boards where there is a signifi- 
cant gap between report-card and EQAO results, 
the results are examined at the school level by 
teachers and principals. There could be a num- 
ber of reasons for this gap, including differences 
between the two assessment processes and 
specific circumstances pertaining to the student 
and family. Such situations are most effectively 
assessed in the classroom, where the needs of 
individual students can be addressed. 


ONTARIO STATISTICAL NEIGHBOURS 
INFORMATION SYSTEM 


The Ontario Statistical Neighbours is an informa- 
tion system for analyzing school performance, 
demographics, and school program information. 
The information in the system comes from three 
main sources: Statistics Canada (2006 Census 
data), the Ministry of Education, and the EQAO. All 
publicly funded schools in the province with ele- 
mentary enrolment are included in the database. It 
is intended to help the Secretariat with its strategic 
planning and to help identify similar schools to 


facilitate the sharing of best practices. It supports 
strategic planning, capacity building, and program 
development, and provides data to help schools 
and boards make decisions based on research and 
evaluation. Ontario Statistical Neighbours does not 
rank schools, nor does it identify any specific indi- 
vidual, student, teacher, or principal. 

This system provides the Secretariat with useful 
information for decision-making and monitoring. 
For example, the Secretariat can relate school per- 
formance information with contextual data, such 
as the percentage of students living in low-income 
households, students whose first language learned 
at home is different from their language of instruc- 
tion, and students with special education needs. 

Eight elementary school principals whom we 
interviewed told us that information from Ontario 
Statistical Neighbours is useful for planning and 
for developing school improvement plans and 
strategies for improving student achievement. How- 
ever, the principals’ use of the tool was infrequent 
because the principals did not have direct access 
to the system—instead, they had to go through the 
time-consuming process of getting the information 
from their student achievement officers. 

We noted that a number of school boards have 
therefore developed their own systems to gather 
information that is similar to that found in Ontario 
Statistical Neighbours. In our view, given that there 
are 72 school boards in the province, enabling 
school boards to have direct access to Ontario Sta- 
tistical Neighbours for planning purposes could be 
more economical than the boards having to develop 
and maintain individual systems. 





RECOMMENDATION 5 





To ensure that all school boards and schools 
can obtain useful and relevant information 

to develop strategies for improving student 
achievement, the Ministry of Education should 
consider granting them direct access to the 
Ontario Statistical Neighbours information 
system. This would be more cost-effective than 


Literacy and Numeracy Secretariat , 201 


school boards having to develop and maintain 
their own systems. 


The Ministry agrees with this recommendation. 
For the past two years, school boards have been 
able to access the Ontario Statistical Neighbours 
database by contacting the Ministry. Surveys 
and focus groups with school board senior staff 
and principals have informed the Ministry that 
the database would be very useful to super- 
intendents of schools and of some interest to 
principals. As a result, the Ministry has been 
working for the past year to develop an online 
version of the database that would give school 
boards direct access to the information. 
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Background 


OntarioBuys is a government initiative launched in 
2004 to achieve savings in the procurement of goods 
and services in the broader public sector (BPS), 
especially by hospitals, school boards, colleges, 

and universities. There is no specific legislation or 
regulation under which OntarioBuys operates. The 
BPS Supply Chain Secretariat, part of the Ministry of 
Finance (Ministry), is responsible for administering 
and managing OntarioBuys. In March 2009, the 
government announced in the provincial Budget 
that it would introduce legislation to expand the 
program’s mandate to include other sectors. 

While certain other provinces may provide 
funding to improve a specific component of the 
supply chain, Ontario is the only province in Canada 
with a formal program that provides funding and 
advice to BPS organizations to help them improve 
their supply-chain-management practices for 
the entire supply-chain spectrum. Specifically, 
OntarioBuys encourages BPS organizations 
to engage in collaborative ordering, delivery, 
warehousing, and payment for goods and services. 
It also funds projects that propose cost-saving 
improvements in other administrative processes, 
such as human resources, payroll, and financial 
management. According to the 2008 Ontario 
Budget, the goal of OntarioBuys is to reduce the 
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time and money spent by the BPS on procuring 
goods, and funnel savings back into front-line 
services. 

Although OntarioBuys commenced its 
operations in the 2004/05 fiscal year, its first full 
year of operation was 2005/06. Since 2004/05, 
OntarioBuys has provided funding of about 
$148 million for two areas. About $88 million has 
been paid for the formation and/or expansion 
of collaborative groups called “shared-service 
organizations” (SSOs). An SSO is a centralized 
organization that BPS institutions join as members. 
The SSO acts on behalf of its members to obtain 
better prices for goods and services through group 
purchasing. The SSO may also serve its members by 
developing more efficient purchasing practices and 
making other collaborative arrangements, such as 
establishing centralized warehouses, distribution 
systems, and information systems. The remaining 
$61 million has been used to fund 53 projects aimed 
at helping BPS institutions become more efficient 
and effective in their supply-chain and other back- 
office processes. 

The BPS Supply Chain Secretariat has increased 
its staffing from 17 people in 2007/08 to 31 in 
2008/09 and engaged external consultants to assess 
BPS business-case proposals for funding of projects 
over $1 million. 

Figure 1 shows OntarioBuys expenditures for 
the five years ended March 31, 2009. 


a 
Figure 1: OntarioBuys Total Expenditures, 
2004/05-2008/09 ($ million) 


Source of data: OntarioBuys 
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Audit Objective and Scope 





The objective of our audit was to assess whether 
OntarioBuys had adequate systems and procedures 
in place to: 

@ ensure that projects were approved and 
delivered in accordance with program 
objectives, policies, and funding agreements; 
and 

@ measure and report on OntarioBuys’ 
contribution to improving the 
cost-effectiveness of government and BPS 
services to Ontarians. 

The scope of our audit included review and 
analysis of relevant files and administrative 
procedures and interviews with appropriate 
OntarioBuys staff, senior management of 
three other ministries—Health and Long-Term 
Care, Education, and Training, Colleges and 
Universities—as well as senior management 
at shared-service organizations and at various 
broader-public-sector institutions such as hospitals, 
universities, colleges, and school boards. We 
also met with the Ontario Hospital Association 
and interviewed senior management of various 
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local, regional, and national group purchasing 
organizations. In addition, we reviewed relevant 
audit reports issued by the Ministry’s internal audit 
services. Wherever possible, we relied on their 
audit work to reduce the extent of our audit. 


In March 2009, the government announced in 

its Budget that OntarioBuys had helped broader- 
public-sector (BPS) entities redirect $45 million in 
savings toward front-line services. All $45 million 
came from the hospital sector, of which $20 million 
was from one shared-service organization (SSO). 
However, almost all of this $20 million in savings 
was kept by the SSO and was not redistributed 

to the hospitals to provide front-line services. 
Instead, most of the savings were retained by the 
SSO for developing information technology for its 
back-office processes. The balance of the reported 
savings came from a number of projects; however, 
OntarioBuys did not verify these savings nor was it 
able to demonstrate that the reported savings had 
actually been invested in front-line services. Our 
review indicated that for a number of projects, the 
savings figures could not be substantiated. 

We acknowledge that OntarioBuys has 
undertaken significant efforts to promote its 
collaborative supply-chain initiatives. BPS entities 
that received OntarioBuys funding advised us that 
the additional resources provided have enabled 
them to focus more attention on the supply-chain 
area. Nevertheless, participation is currently well 
below the level required for OntarioBuys to achieve 
its goals, particularly in the education sector. Our 
specific findings are as follows: 

@ The province has spent about $58 million to 
fund the formation or expansion of nine SSOs 
in the health sector and $30 million for one 
SSO in the education sector. At the time of 
our audit, 50% of Ontario hospitals with 70% 
of Ontario’s total beds were participating in 
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health SSOs. There was limited participation 
in the education SSO, which may be due 

in part to the fact that various educational 
institutions had initiated a number of their 
own purchasing consortia over the years. 

@ Since 2005, the education SSO has received 
$30 million in approved OntarioBuys 
funding. The SSO had committed to sign 
up 13 of the province’s 116 school boards, 
colleges, and universities as well as 1,000 
suppliers by June 2009 to participate in a 
new electronic purchasing system called 
e-Marketplace. However, the e-Marketplace 
had yet to become operational by June 
2009, and no institutions had formally 
signed up to be members. Subsequent to 
our audit, OntarioBuys informed us that 
the e-Marketplace would be operational by 
October 2009. 

We noted three operational areas at OntarioBuys 
where improvements are required—the review of 
business cases submitted for funding approval, the 
monitoring of funded projects for achievement of 
contract deliverables, and competitive procurement 
processes: 

e@ OntarioBuys bases its funding approvals 

for SSOs on business cases prepared by BPS 
organizations. The business cases are to 
include estimated costs and potential savings, 
and OntarioBuys engages consultants to 
review on its behalf all business cases for 
projects with funding of over $1 million. 

The underlying business case for the 
education-sector SSO—the largest SSO 
funded—projected that its collaborative 
purchasing and e-Marketplace initiatives 
would yield benefits/savings of $669 million 
over five years. This included total savings to 
group members of $294 million through their 
use of the e-Marketplace and $375 million 
from group purchasing. However, our review 
of the business case, file documentation, 

and external consultants’ reports found that 
the estimated savings were often based on 


unreasonable assumptions. We also found 

no evidence that three-quarters of the 

project risks identified had been resolved. 
OntarioBuys’ management told us that the 
issues had been discussed and addressed, but 
there was no documentation to show how the 
risks, including a number of high risks, were 
resolved. 

Similarly, with respect to the $61 million 
spent on projects for improving supply-chain 
and back-office processes, evidence was 
lacking that projected costs and savings were 
appropriately assessed. For example, one 
project’s projected savings were based ona 
hospital’s extrapolating the results from its 
emergency unit to the entire hospital without 
any evidence to support the reasonableness 
of such a projection. For another project, 

the estimated costs submitted for funding 
were revised three times over a four-month 
period, from $455,000 to over $1 million. 
The amount of OntarioBuys funding was 
based on a percentage of estimated costs. 
However, we found no documentation on file 
to show that either the cost revisions or the 
projected savings had been properly assessed. 
OntarioBuys indicated that these projects 
were approved prior to April 2007 when it did 
not have sufficient staff. 

Once projects were approved and funds 
provided, OntarioBuys did not have program- 
specific guidelines for consistent and effective 
monitoring of their progress. There were no 
program-specific guidelines for conducting 
site visits, documenting work performed, 
verifying deliverables prior to the release of 
final payments, and closing files for completed 
projects. We noted that some files contained 
detailed review notes with good supporting 
documentation to verify the reported project 
status. However, many others did not. 
OntarioBuys indicated that it had hired more 
staff in 2008/09 to strengthen the monitoring 
processes. As a result of the insufficient 
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number of staff devoted to monitoring in 
will be used to improve OntarioBuys to ensure it 


delivers value for money to Ontarians. 

Ontario’s broader-public-sector (BPS) organ- 
izations spend more than $10 billion annually 
acquiring the goods and services needed to 
deliver health care, education, and other vital 
public services. This substantial and complex 


the earlier years and the lack of program- 
specific guidelines, projects—especially those 
approved before 2008—were not consistently 
and effectively monitored. 

e In response to our request for a listing 
of projects and their related savings that 
made up the total reported savings of 


$45 million announced in the Ontario Budget, expenditure of public money, cumulatively as 


OntarioBuys provided a list supporting large as the supply chains of some of Ontario’s 
the project savings up to the end of March 
2009. Our review of a sample of projects 

from this list, which accounted for 75% of 
the $45 million, indicated that, other than 


for one project with savings of $20 million, 


largest corporations, needs to be efficient and 
effective for proper support of front line servi- 
ces and for BPS organizations to satisfy their 
accountability obligations. 

As the Auditor General has noted, Ontario 
is the only province with a formal program to 


the reported savings provided to us were at os : ' 
; help BPS entities improve their practices for 
questionable. For example, savings totalling : ; 
en the entire supply chain spectrum. OntarioBuys 
$7.3 million were reported for two projects 


for the fiscal years from 2006/07 to 2008/09, 
yet these two projects were supposed to 


has undertaken significant efforts to promote 
integrated supply chain leading practices, 
dominantly th h the introduction of it 
have been completed by December 2006. In ae ea eett . ee Rit oko DM a 
Supply Chain Guideline. 
We appreciate the Auditor General’s efforts 


to help improve OntarioBuys to ensure it deliv- 


fact, neither project was completed at the 
time of our audit. Subsequent to our review, 


OntarioBuys advised us that there were other : me 
é ; or ers value for money to Ontarians. The Ministry 
savings not included on the original list and ; ’ 
' : : will use the recommendations of the Auditor 
that, notwithstanding the concerns we raised, ; ; 
General’s report as a basis for moving forward 
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the reported total savings of $45 million 





with the OntarioBuys program. The Ministry is 
constituted a reasonable estimate. % ; : 

; already working to improve documentation of 

e According to OntarioBuys, the SSOs and on 5 
SOR j } j the program’s internal business processes and 

BPS organizations involved in the projects 


have spent about $45 million of the funding 
provided to them since the 2004/05 fiscal 
year to hire some 270 consultants for a 


reported savings, and to monitor its funded 
projects more closely. The Ministry recognizes 
that some areas need to be strengthened, and 


the recommendations will help us do so. 


variety of reasons. We reviewed a sample of 
consulting contracts totalling $15 million 
from various projects and found that over 
40% did not comply with the competitive 


procurement requirements of the project Detailed Audit Observations 


funding agreements. 


BGUERACIMINISTRY RES Ose EO eine eneeaTION 


As a new initiative, OntarioBuys has made 





The Ministry welcomes the review and findings 


; considerable efforts to publicize and promote 
of the Auditor General. The recommendations 


its program to other government ministries and 
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the broader public sector. Promotional activities 
conducted by OntarioBuys designed to increase 
BPS participation in the program included: 

e@ holding joint seminars and conferences with 
the Ontario Hospital Association on supply- 
chain-management practices; 

e@ attending and presenting educational sessions 
on supply-chain-management practices within 
the health-care sector; 

@ meeting with various universities, colleges, 
and school boards in Ontario to promote the 
OntarioBuys program; 

e consulting with BPS sectors and obtaining 
feedback on the supply-chain guidelines that 
it was developing; 

e working with other ministries to promote the 
OntarioBuys program to their respective BPS 
organizations; and 

e informing and educating suppliers on the BPS 
initiatives funded by OntarioBuys. 

Our interviews with senior management at 
other ministries as well as with staff at various 
health and education associations showed that they 
generally supported the OntarioBuys program. 

One significant communications component 
of OntarioBuys is the dissemination of leading 
practices and the sharing of information to help 
BPS members spend their money more effectively. 
In this regard, OntarioBuys has published a 
supply-chain guideline; a compendium of leading 
practices in integrated supply-chain management; 

a report on supply-chain modernization in Ontario 
health care; sample business-case templates for 
BPS organizations to use; and performance- 
measurement-related documents for the hospital 
sector. 

OntarioBuys informed us that, following the 
recommendations of a 2007 consultant’s report, 
it plans to improve its website to enable increased 
dissemination of the above-noted documents as well 
as enhanced communication among OntarioBuys, 
its projects, and the broader public sector. 

As of the time of our audit, OntarioBuys had 
developed annual communication and promotion 


plans to focus on objectives for the coming year, but 
it had not developed any long-term communication 
and promotion plans to address the program’s 
overall goals. OntarioBuys informed us that, prior 
to the March 2009 Budget, it developed only short- 
term plans because the program was an annual 
budget initiative that could have been terminated at 
any time. 

The 2007 consultant’s report also indicated 
that communications were not performed to a 
“desired” level and that existing staffing had limited 
additional communications capacity. In response 
to this, the BPS Supply Chain Secretariat in late 
2008 established a new unit with designated staff 
to concentrate on communications activities, 
including the development of a long-term 
communications plan. 


APPROVAL OF REQUESTS FOR FUNDING 


As of March 31, 2009, OntarioBuys had disbursed 
about $88 million for the development or expan- 
sion of SSOs. The funding enabled these SSOs 

to provide their members with wide-ranging 
assistance such as group purchasing, centralized 
warehousing and distribution, and implementation 
of management information systems so that their 
supply chains are better managed. OntarioBuys had 
also disbursed about $61 million for the develop- 
ment and implementation of projects focused on 
improving specific areas of the supply chain for 
individual institutions. Examples of such projects 
included electronic cataloguing, warehouse auto- 
mation, and research-data sharing. According to its 
guidelines, OntarioBuys can fund up to 100% of the 
costs that parties applying to become an SSO incur 
as they put together their SSO proposal, including 
all the costs of background analysis and prepara- 
tion, up to 75% of the costs of actually forming the 
SSO, and up to 85% of the costs of the projects for 
improving the supply chain of individual institu- 
tions. Figure 2 gives an overview of the funding 
OntarioBuys has approved and funded for these 
purposes over the last five years. 


Ca a 
Figure 2: OntarioBuys Funding, Approved and Paid, 
2004/05-2008/09, ($ million) 


Source of data: OntarioBuys 
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OntarioBuys approves funding requests— 
whether for SSOs or for supply-chain-improvement 
projects—on the basis of a review and evaluation 
of the applicant’s business case. A business case 
typically includes a description of the proposal’s 
goals, expected benefits, and estimated costs, as 
well as an implementation strategy. OntarioBuys 
may request additional information or support 
during the course of the review. For proposals 
requesting more than $1 million, OntarioBuys 
engages external consultants to review the business 
case and to identify significant risks or concerns. 
The Ministry of Finance’s internal audit staff also 
conduct a general review of the business cases of 
proposals requesting more than $1 million. If the 
review leads to approval, the applicant must sign a 
transfer-payment agreement that sets out the terms 
and conditions for funding. 

Our observations with respect to requests for the 
funding of both SSOs and individual supply-chain- 
improvement projects are in the following sections. 


SSO Requests 


Since its inception, OntarioBuys has approved and 
funded business-case requests for 10 SSOs—nine in 








OntarioBuys Program 


the hospital sector and one in the education sector. 
These SSOs are expected to be self-sustaining, 
primarily through savings generated from their 
members’ obtaining lower prices for their purchases 
and implementing more efficient supply-chain 
practices. 

To assess the adequacy of the OntarioBuys 
review-and-approval process for requests for SSO 
funding, we examined the business cases and pro- 
ject files with the largest projected savings and costs. 


Health-sector SSOs 
As of March 31, 2009, OntarioBuys had disbursed 
about $58 million for the formation of nine health- 
sector SSOs, as shown in Figure 3. Four of the SSOs 
existed before the establishment of OntarioBuys. 
OntarioBuys funding enabled these four organiza- 
tions to expand and include additional institutions. 
Prior to the formation of these SSOs, Ontario hospi- 
tals and other health-care-sector BPS organizations 
relied mainly on two national purchasing groups to 
meet their collaborative purchasing needs. 

We reviewed the proposal for the largest 

SSO, “A”. As Figure 3 indicates, prior to the estab- 
lishment of OntarioBuys, 18 hospitals from two 
separate working groups were exploring in 2003 
the potential savings from collaborative purchasing 
and other supply chain functions. Subsequently, 
these hospitals decided to merge into one group 
to review these areas for potential savings. When 
OntarioBuys invited applications for the formation 
of SSOs, this group of hospitals applied for funding 
in December 2004 to form an SSO. OntarioBuys 
approved funding of $22 million for this SSO in 
February 2005 and added another $2 million in 
funding as a result of changes to the SSO agree- 
ment. By March 2006, six of these hospitals had 
decided to join other SSOs or opted to buy supplies 
through national purchasing groups. We found 
that the costs that the proposed SSO stated for 
its business case and the savings estimates were 
reasonably supported. The SSO has since generated 
savings and become self-sustaining. 
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Figure 3: Funding for Health-sector SSOs 


Source of data: OntarioBuys 


Funding Pre-existing / 
($ million) — oe 


1. Before the OntarioBuys program began, the SSO that became “A” 
consisted of two working groups that subsequently merged in 2004. 


2. “H” was proposed as an SSO for the northwest and northeast areas 
of the province. The proponent was provided with $1.7 million to 
assess the feasibility of the SSO for this region and concluded that it 
would not be viable. Subsequent to our audit, OntarioBuys informed 
us that they have recovered $600,000 from the proponent. 


One proposed health SSO covering the north- 
west and northeast parts of the province—‘“H” in 
Figure 3—was found to be not viable after $1.7 mil- 
lion was provided to this proponent to assess the 
feasibility of forming such an SSO. Subsequent to 
our audit, OntarioBuys informed us that $600,000 
of this total has been recovered. 


Education-sector SSO 
The single SSO proposal from the education sector 
was the largest OntarioBuys initiative in terms of 
projected benefits/savings—$669 million over five 
years. This included estimated savings to group 
members of $294 million on implementation of 
an electronic e-Marketplace purchasing site and 
$375 million from group purchasing. Our review of 
the business case, file documentation, and external 
consultants’ reports found that OntarioBuys’ 
approval of a total of $41 million in funding was 
based on questionable assumptions about savings 
and the level of participation of suppliers and 
educational institutions. Specifically, we found the 
following: 


e An external consultant was engaged in 2007 


to review and validate the expectations, 
assumptions, and methodology used in 
preparing the proposal, including the financial 
model (operating revenues, expenses, and 
balance sheet), implementation timetable, 
resource plan, and the anticipated benefits 
from e-Marketplace and group purchasing. 
Although the consultant’s report did not 
address the reasonableness of the specific 
assumptions used, it did identify over 200 
risks relating to this project. For example, one 
risk was low supplier participation because 
most suppliers were not e-Marketplace ready 
and would have to invest heavily in resources 
to connect with the system. Furthermore, 

we found that follow-up action was taken for 
only about 50 of these over 200 project risks. 
OntarioBuys informed us that it discussed 
and addressed the remaining 150 risks—but 
there was no documentation to show how 

the risks were resolved. A number of them 
were clearly identified as high or medium 
risks by an external consultant. For instance, 
the consultant recommended that supplier 
readiness be assessed to mitigate against the 
risk of low supplier participation, but this 
assessment had still not been performed at the 
time of our audit. 

In contrast to projected savings to be achieved 
by the largest health-sector SSO, which were 
based on participation by only 12 members, 
the projected savings to be achieved by the 
education-sector SSO required that 116 
educational institutions participate fully both 
in group purchasing and in e-Marketplace. 
However, 65% of Ontario’s 44 colleges and 
universities and 80% of Ontario’s 72 school 
boards were already participating in various 
local purchasing collaborative groups. 
OntarioBuys had not realistically considered 
the likelihood that these institutions would 
not fully participate in both group-purchasing 
and e-Marketplace services. In fact, no 


institutions had even signed up as members of 
e-Marketplace at the time of our audit. 

@ The $375 million in projected savings from 
group-purchasing activities was determined 
arbitrarily. On the one hand, the business case 
extrapolated this amount on the basis of 116 
member institutions spending $4.4 billion on 
purchases. On the other hand, the minutes of 
an SSO board of directors meeting indicated 
that the same $375-million figure was arrived 
at on the basis of $3.3 billion in purchases. It 
appeared that the business case essentially 
worked back from the $375 million of savings 
to arrive at the necessary expenditures 
required to generate the projected savings. 
Subsequent to our audit fieldwork, 
OntarioBuys provided us with a new group- 
purchasing-savings estimate of $113 million 
over five years, a 70% decrease from the 
original projection but still more than the 
amount of funding OntarioBuys originally 
provided to the SSO. 

e The methodology used in the education 
sector’s proposal to calculate the $294 million 
in estimated process savings from the use of 
e-Marketplace was based on extrapolating 
one university’s 2004 electronic procurement 
savings to 116 institutions. We found no 
evidence that OntarioBuys staff or its 
consultants assessed the reasonableness of 
this methodology or validated the data used. 
OntarioBuys staff indicated that they also 
found the savings of $294 million provided 
in the SSO business case to be unreasonable, 
which is why they did not include the 
$294 million in their analysis of the business 
case. However, the analysis itself did not 
explain why the $294 million had been 
excluded. 


Project Requests 


In addition to reviewing OntarioBuys’ approval 
process for SSO requests for funding, we also 
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reviewed its approval process for a sample of 
requests for funding for projects to improve the 
supply-chain practices of individual institutions. We 
found inconsistencies in OntarioBuys’ evaluations 
and related documentation, as well as insufficiently 
documented business-case reviews. It was left up to 
individual OntarioBuys staff reviewers to follow up 
and document the information they deemed to be 
important, but there was no documented evidence 
of supervisory oversight to ensure that significant 
concerns were being satisfactorily addressed. The 
following are some examples: 

@ One project involved the implementation of 
an information system to allow educational 
institutions and students to access research 
sites. The project revised its estimated costs 
numerous times, from an initial $455,000 to 
a final estimate of over $1 million. However, 
our review showed no documentation on file 
to validate the various cost revisions. Project 
review staff told us that verbal discussions 
did occur with the project managers, but we 
found no documented explanation of the 
reasonableness of these cost revisions. The 
same project estimated 40% in cost savings 
from collective purchasing, or more than 
$500,000 annually. However, we found no 
documentation that OntarioBuys assessed 
the reasonableness of this estimate when 
approving its additional investment. 

e@ For another project that received $1 million in 
OntarioBuys funding, an external consultant 
noted that the project’s estimates for the 
cost of implementation were “deliberately 
conservative” and that implementation 
would likely cost 30% less. However, we 
found no documentation of any follow-up to 
resolve the issue. Because projects are funded 
on the basis of a percentage of projected 
costs, overestimating costs could result in 
OntarioBuys distributing excessive amounts 
of funding to these projects. We also noted in 
this regard that OntarioBuys had no program- 
specific guidelines for recovery of overfunded 
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amounts. Subsequent to our audit, 
OntarioBuys advised us that it has initiated 
recovery actions on three projects and has 
received funding back from two of them. 

e Another project extrapolated the savings 
achieved from reductions in staff time and 
inventory in a hospital emergency room to the 
entire hospital. OntarioBuys did not request 
the emergency-unit data to verify the results, 
and we found no documentation to show that 
OntarioBuys assessed the reasonableness of 
this extrapolation. 

We noted that there was no central tracking of 
the comments and issues that reviewers identified in 
their project reviews of each file. This made it much 
more difficult to determine whether issues had been 
addressed or whether action was still required. 

OntarioBuys acknowledged that supervisory 
oversight was not well documented in the years 
prior to 2007/08. According to OntarioBuys, 
as more staff have been hired, the supervisory- 
oversight process and related documentation have 
become more rigorous. It has also developed a 
centralized summary template tracking reviewers’ 
comments that is to be used for project reviews. 


RECOMMENDATION 1 





To ensure that estimated costs and benefits in 
business cases are appropriately assessed before 
being approved, OntarioBuys should: 

e obtain the necessary supporting materials 
from applicants to appropriately assess the 
reasonableness of projected savings and 
estimated costs; and 

e address identified risks and document actions 
taken or to be taken to mitigate these risks. 





OntarioBuys will develop guidelines to codify 


and strengthen current practices with respect 
to the assessment of savings and costs by Janu- 
ary 31, 2010, and will provide mandatory train- 
ing to staff once these guidelines are developed. 


OntarioBuys will develop guidelines on the 
identification, documentation, and tracking of 
project risks by November 30, 2009, and will 
provide mandatory training to staff once these 
guidelines are developed. 


MONITORING OF FUNDED PROJECTS 
AND EXPENSE CLAIMS 


Oversight of the Status of SSOs and 
Projects and the Achievement of 
Deliverables 


The Ontario government requires that OntarioBuys 
comply with the Transfer Payment Accountability 
Directive in its transfer payment agreements with 
the BPS for funded projects. The Directive requires 
that, once transfer payment agreements are signed, 
OntarioBuys have the oversight capacity to ensure, 
through ongoing project monitoring on a timely 
basis, that projects are providing the services for 
which the funds were received. Transfer payments 
must be monitored after disbursement to ensure 
that all contracted conditions and deliverables are 
being met. When recipients fail to meet the transfer 
payment conditions, the unspent funds must be 
assessed then recorded as an accounts receivable 
balance, as required by the government’s Transfer 
Payment Recovery Operating Policy. 

Although OntarioBuys is required to comply 
with the above directives, we found that it had 
not developed program-specific guidelines for 
consistent and effective monitoring of transfer 
payments. Specifically, no guidelines existed 
outlining requirements such as the level and 
type of documentation needed to support the 
review conducted; timeliness and extent of the 
review and monitoring of the project status 
and required achievements; management of 
payments; timeliness and frequency of site visits 
and work required, if any, during the site visits; and 
procedures to be performed prior to the release of 
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the final project payments and the closing of files Subsequent to our audit, the reported savings 
for completed projects. of $4.8 million were revised downward to 

In general, the level of documented monitoring $1.1 million. 
varied among project files. We did note that some e@ OntarioBuys reported cumulative total 
files contained detailed review notes with good savings for another project of $2.5 million 
supporting documentation to verify the reported ($820,000 annually) over a three-year period 
project status. However, many others did not. For from the 2006/07 fiscal year to 2008/09. 
instance: This initiative, also related to electronic 

@ At the time of our audit, OntarioBuys reported purchasing, was to have been completed 


cumulative savings of $4.8 million over three 
years from an electronic purchasing project 
that had a completion date of December 31, 
2006. Our audit showed that OntarioBuys 
had committed $2 million to the project 

and that the final payment of $600,000 was 
made in March 2007. We found no evidence 
that OntarioBuys verified that the funds 
were spent in accordance with the project’s 
funding agreement and that the project had 
actually been completed before the release 
of the final payment. When we approached 
the BPS organization’s project management 
to verify the reported savings, they informed 
us that the project was still not completed. 
Our discussion with OntarioBuys staff 
indicated that they learned of the project’s 
incomplete status only when a new project 
manager informed them in February 2008. 
Furthermore, OntarioBuys staff informed 

us that they had verbally approved a project 
completion extension to September 2008. 
However, other than reporting the above 
$4.8 million in achieved savings, OntarioBuys 
undertook no formal review of the project’s 
status after granting the verbal extension. 
When we inquired about that status, 
OntarioBuys staff visited the project site and 
found that $636,000 of its funding was still 
unspent, with $94,000 in interest adding 

to it. When we followed up with project 
management in May 2009, they informed us 
that the funds had still not been spent and 
that the expected project completion date 
had been revised again to later this year. 


by December 31, 2006. However, when we 
approached the BPS organization’s project 
Management, the manager told us that the 
project was not completed and that he was 
unsure how the savings number was arrived 
at because no baseline had been established 
against which potential savings could be 
measured. Approved funding for the project 
was $1.7 million, and total payments made 
up to March 2007 amounted to $1.3 million. 
The balance was to have been released later 
in 2007 when the project was to have been 
completed. However, at the time of our audit, 
the project was still ongoing, and we found 
no documents relating to an extension. After 
we raised the issue with OntarioBuys, the 
reported savings of $2.5 million were revised 
downward to zero. 

@ Between 2005 and 2008, OntarioBuys 
disbursed a total of $6 million to the 
education-sector SSO to fund its development 
of an implementation plan and business case. 
The funds were paid through three separate 
agreements and amendments to those 
agreements. But only for the first agreement 
did OntarioBuys request an expenditure 
report for actual spending. OntarioBuys did 
not request any final actual expenditures for 
the other two agreements to assess whether 
there were unspent funds. 

Our audit also found that OntarioBuys staff 
were manually monitoring their assigned projects 
through tools such as Excel spreadsheets and indi- 
vidual file notations. An information system to help 
staff track individual project progress, required 
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deliverables, the achievement of deliverables, 
outstanding information to be reviewed, and the 
progress of payments need not be complex nor 
costly. Ontario Buys informed us that it was in the 
process of developing an information system to 
address this concern. 





RECOMMENDATION 2 


To ensure that the shared-service organizations 
(SSOs) and projects that OntarioBuys funds 
achieve contract deliverables and that funds 
are used for the intended purpose, OntarioBuys 
should: 

e develop monitoring guidelines to assist its 
staff in consistently conducting appropriate 
oversight of the SSOs and projects funded; 
and 

@ monitor, on a timely basis, the progress of 
funded SSOs and projects against contract 
deliverables and take appropriate action 
when there are significant delays. 





OntarioBuys will develop guidelines to codify 


and strengthen its current project-monitoring 
practices by December 31, 2009, and will 
provide mandatory training to staff once these 
guidelines are developed. OntarioBuys is 
upgrading to an electronic transfer payment 
tracking system, which it expects to be com- 
pleted in December 2009. 

OntarioBuys will provide mandatory train- 
ing to staff on the TP Accountability Directive, 
including modules on the assessment of busi- 
ness cases and project monitoring. 


Consulting Services 


Funding agreements for all OntarioBuys SSOs 
and projects include a copy of the OntarioBuys 
procurement policy guidelines. SSOs and projects 


are required to comply with these guidelines when 
acquiring services, including those of consultants. 

The guidelines state that SSOs and projects must 
use an appropriate level of competition to obtain 
the best value for funds to be spent on consultants. 
In particular, if the estimated contract value is 
$25,000 or more, but less than $100,000, SSOs 
and projects must invite at least three potential 
vendors to submit written quotes and proposals. 

If the estimated contract value is greater than 
$100,000, an open and transparent public Request 
for Proposal (RFP) process must be followed. 
Exceptions to these competitive requirements must 
be approved in writing by the Ministry. 

According to OntarioBuys, the SSOs and the 
BPS organizations involved in funded projects since 
2004/05 have spent about $45 million for some 
270 consultants. However, OntarioBuys did not 
maintain information on these consultants, such 
as the purposes for which they were hired, the 
contract rates and amounts, and the subsequent 
payments made. Upon our request, OntarioBuys 
prepared a list of consultants from information 
gathered from the various SSOs and projects. 
Using this list, we reviewed a sample of consulting 
contracts totalling $15 million from various SSOs 
and projects and found the following: 

e@ Almost half of the contracts we reviewed did 
not comply with the guidelines for obtaining 
competitive quotes. Predominantly, single- 
sourcing was used to contract for consultant 
services. According to the guidelines for 
significant contracts, single-sourcing is 
allowed only if a consultant has specific 
knowledge that cannot be provided by any 
other party, if it is an urgent situation, or if 
only one vendor can provide the requested 
services. In addition, approval for single- 
sourcing must be obtained in writing from 
the Ministry. In the cases noted above, we 
found no written approval to allow for single- 
sourcing. 

e For 40% of the completed consulting 
contracts, the total payments exceeded the 


contract ceiling price, and most of these 
consultants continued to be paid beyond 
the contract period. We found no evidence 
of amended contracts to support the final 
payment amounts or the extension of the 
payment period. 

OntarioBuys staff said that they were also 
concerned about SSOs and projects not complying 
with the procurement guidelines, especially 
given that BPS project managers had all signed 
certificates indicating compliance. In fact, they 
noted that one project single-sourced $1.1 million 
of $2.6 million in consultant contracts without 
prior approval. When this was discovered in late 
2007, OntarioBuys requested that this project 
submit documentation supporting the decisions 
relating to consultant contracts for a six-month 
period. OntarioBuys staff indicated, however, that 
owing to resource constraints, they did not review 
the contracts for other SSOs and projects to ensure 
that procurement policies had been complied with. 
OntarioBuys recognized that this was a high-risk 
area and requested that the Ministry’s internal 
audit services review this area in 2007/08. In 
its 2007/08 audit plan, internal audit services 
included plans to visit BPS institutions to determine 
whether funding provided was monitored and 
the terms and conditions of the transfer payment 
agreements complied with. However, owing to 
staffing requirements in other areas, including 
providing assistance to our Office on the annual 
Public Accounts audit, the work was postponed to 
2008/09. As a result of our review of OntarioBuys 
in 2008/09, internal audit decided to delay this 
work until 2009/10. 


RECOMMENDATION 3 


To ensure that significant consulting-service 
contracts are awarded in an open, fair, and 
transparent manner, OntarioBuys should 
monitor broader-public-sector compliance with 
the required procurement policies. 


OntarioBuys Program SEES: 


OntarioBuys will create project procurement 
monitoring guidelines by November 30, 2009, 
and will provide mandatory training to staff 
once these guidelines are developed. 

OntarioBuys will inform projects about 
increased monitoring by November 30, 2009, 
and will implement the guidelines by Febru- 
ary 1, 2010. 


Review and Approval of Expense Claims 


When conducting project work, project staff and 
their contracted consultants often incur expenses 
relating to travel, meals, and hospitality as well 

as other activities. Claims for these expenses 

are reimbursed from project funds provided by 
OntarioBuys. We noted that OntarioBuys did not 
provide SSOs or the management of supply-chain- 
improvement projects with a policy or guideline 
on expense-claim reimbursement. We found cases 
where, as a result, SSOs reimbursed staff of BPS 
organizations for expenses that would not be eli- 
gible for reimbursement under government policy 
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or, if they were, exceeded the maximum amounts 





allowed under government policy. For example: 

@ Several projects reimbursed staff for numer- 
ous meal claims for amounts that were con- 
siderably more than the maximum amount 
allowed for Ontario government employees. 

@ An SSO reimbursed the cost of a dinner to 
celebrate the signing of an agreement to build 
an information system. 

e@ The same SSO reimbursed the cost of a second 
celebration, held in the office, relating to the 
same agreement. 

e AnSSO reimbursed the cost of sending flower 
bouquets to the homes of each of its staff 
members (including contract staff) in appre- 
ciation of work done. 

We also noted that OntarioBuys had not 

developed program-specific guidelines for the 
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required level of review of these expenditures by 
its staff. 


RECOMMENDATION 4 


To ensure that only appropriate expenses are 
reimbursed, OntarioBuys should provide the 
management of shared-service organizations 
and supply-chain-improvement projects with 
guidelines on the reimbursement of meal, 
travel, and hospitality expenses, with maximum 
limits that are reasonable when compared to 
those for Ontario government employees. 


OntarioBuys will develop a guideline on project 
expense claims by November 30, 2009. Once 
developed, existing projects will be advised that 
compliance is required and new projects will 
have the guideline incorporated directly into 
their TP agreements. 


PERFORMANCE MEASUREMENT 
BPS Participation in SSOs and Projects 


According to its mission statement, OntarioBuys is 
“to facilitate and accelerate the widespread adop- 
tion of integrated supply-chain and other back 
office leading practices by Ontario’s Broader Public 
Sector.” After four years of operation, OntarioBuys 
has not made major headway in facilitating the 
adoption of integrated supply-chain practices in 
BPS sectors other than at hospitals. About 50% of 
Ontario hospitals are members in the eight health 
SSOs and involved in supply-chain-improvement 
projects. According to OntarioBuys, these hospitals 
represent about 70% of Ontario hospital beds. 
With respect to the education sector, the one 
education SSO had not signed up any members 
for e-Marketplace at the time of our audit, and it 
had not finalized any group purchasing contracts. 
The following section relates the performance of 
OntarioBuys as it involves this SSO. 


Education-sector Participation 
OntarioBuys has funded one SSO in the education 
sector, paying out $30 million since 2005. This 
SSO was formed to achieve two primary object- 
ives: develop an electronic purchasing site called 
“e-Marketplace” and facilitate group purchasing for 
the education sector. Provided that the SSO signed 
up a sufficient number of member institutions and 
suppliers, membership fees and supplier and pur- 
chaser transaction fees were projected to generate 
sufficient revenues to enable this SSO to become 
self-sustaining. According to the original March 27, 
2008, agreement with OntarioBuys, the SSO com- 
mitted to the milestones and scheduled completion 
dates for e-Marketplace shown in Figure 4. 

No institutions had signed up as paying 
members for e-Marketplace at the time of our audit, 
and no suppliers could go live on e-Marketplace 
because it was not yet operational. The agreement 
with OntarioBuys was amended in March 2009 
to delay the milestone dates by about a year. 

The SSO told us that it was considering waiving 
the institutional membership fees to encourage 
participation. As well, the SSO informed us that it 
was revising its formal business case. Subsequent 
to our audit, OntarioBuys informed us that it had 
received a revised draft business case in July 2009 
but added that it needed significant revisions. It 
also indicated that the e-Marketplace would be 
operational by October 2009. 

At the time of our audit, the education SSO 
management indicated that no group purchasing 
contracts had yet been finalized, although they 
were working on potential contracts for photo- 
copying machines, photocopy paper, and other 
office supplies. The SSO also was in the process 
of engaging an external consultant to review the 
various options for group purchasing of natural gas. 
Subsequent to our audit fieldwork, OntarioBuys 
informed us that, as of June 30, 2009, 39 educa- 
tional institutions had expressed interest in partici- 
pating in the above group purchasing contracts, but 
no formal arrangements had yet been negotiated. 
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Figure 4: Education SSO Contract Commitments for e-Marketplace to June 2009 


Source of data: OntarioBuys 






Milestones 


outsourcing service agreement with information systems 
service provider 


signing up at least six institutions to use e-Marketplace 
seven additional institutions to agree to use e-Marketplace 
50 suppliers to join the supplier network 


six institutions and 500 suppliers to be “live” (fully active, so 
that institutions can make their purchases) on e-Marketplace 


13 institutions and 1,000 suppliers live on e-Marketplace, 
processing $30 million of purchases 


Our recent audits of Ontario colleges, 
universities, and school boards indicated that 
many of these educational institutions were 
already members of various purchasing groups, 
partnering with other public-sector entities such as 
municipalities and other non-profit organizations. 
For example, our 2006 audit of four colleges showed 
that each of the colleges already participated in 
purchasing groups for goods and services such as 
natural gas, printing and photocopying, cleaning 
services, and paper products. We also noted 
instances where colleges used the prices obtained 
by the purchasing groups of other colleges to get 
a better price from their own suppliers. We noted 
similar examples with school-board purchasing 
consortia that had already been established. 

Our review indicated that the recent group- 
purchasing initiatives of the SsO—such as 
purchasing photocopying machines, paper 
products, and natural gas—were mostly already 
being undertaken by various existing collaborative 
purchasing groups. The fact that most institutions 
in the education sector have already been 
participating in various collaborative purchasing 
groups might help explain their reluctance to 
join the SSO. As well, the significant delay in the 
implementation of the e-Marketplace might also 
have contributed to their reluctance. 


Scheduled 
Completion Date Actual Completion 


June 1, 2008 Aug. 7, 2008 
June 1, 2008 no institutions signed up as of June 2009 
Sept. 11, 2008 no institutions signed up as of June 2009 
Dec. 10, 2008 no suppliers signed up as of June 2009 
Feb. 20, 2009 e-Marketplace not operational as of 

June 2009 
June 4, 2009 e-Marketplace not operational as of 

June 2009 


RECOMMENDATION 5 


To assist Ontario educational institutions 

to more effectively generate savings from 
improved supply-chain-management practices, 
OntarioBuys should more formally assess the 
impact of the various collaborative purchasing 
initiatives already in place in the education sec- 
tor on the effectiveness of the education shared- 
service organization (SSO) and assess whether 
any changes are necessary to the education 
SSO’s business model. 


OntarioBuys will undertake and complete the 
recommended assessment of various collabora- 
tive initiatives and assess their impact on the edu- 
cation SSO’s business model by April 30, 2010. 


Reported Savings 


In March 2009, the government announced in 

its Budget that OntarioBuys had helped BPS 
entities redirect $45 million in savings toward 
front-line services and it stated that these annual 
savings would reach the $100-million level by the 
2011/12 fiscal year. It also announced that a new 
co-ordinated, integrated approach to procurement 
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would result in $200 million in total annual savings 
within the first three years of operation. 

During our audit, OntarioBuys provided us with 
a list of reported project savings to support the 
amount announced in the Budget. For our review, 
we selected four projects with reported savings that 
accounted for over 75% of the $45 million. Of this 
$45 million, nearly half—$22 million—came from 
the largest SSO, providing services to 12 hospitals. 

We asked OntarioBuys to provide support for 

this figure of $22 million and also asked the SSO 
for its audited financial statements. Our review of 
the audited statements showed that, after paying 
for the fees charged by the SSO, a cumulative 
saving of $20 million was available for distribution 
to members as of March 31, 2009. Our review 

also showed, however, that practically all of this 
$20 million remained with the SSO and was not 
redistributed to member hospitals to provide front- 
line services. In fact, only about $337,000 was 
distributed to the 12 hospitals in the 2006/07 fiscal 
year, and nothing was redistributed in 2007/08 and 
2008/09. The SSO informed us that the hospitals 
let it keep the remaining $19.6 million of the 
savings in order to fund the next phase of the SSO’s 
implementation plan, which was to improve its 
information technology for its back-office processes. 

The balance of the $23 million in reported 

savings came mainly from a number of supply- 
chain-improvement projects for individual 
institutions. We reviewed three projects that 
accounted for about $12 million of the $23 million 
and found the following: 

e Our discussions with the management of two 
health projects with reported cumulative 
savings of $2.5 million and $4.8 million, 
respectively, indicated that these figures were 
mainly based on estimates. They said that 
the accuracy of the savings would be difficult 
to ascertain because no baselines had been 
established against which the savings could 
be determined. As noted in the earlier section 
on oversight of projects, these two projects 
were to have been completed in 2006 but 


were incomplete at the time of our audit 

in 2009. We also noted in that section that 
OntarioBuys’ revised list of savings, provided 
to us after we completed the audit, reduced 
the reported savings of $2.5 million to zero 
and $4.8 million to $1.1 million. 

@ OntarioBuys reported that one project saved 
one hospital a total of $4.6 million over three 
years to March 31, 2009. When we asked 
OntarioBuys to provide support for this figure, 
it could support savings of only $1.1 million. 
When we visited the hospital in May 2009 to 
follow up, and hospital management provided 
us with its latest report, which showed that 
cumulative savings to June 2008 totalled 
only $2.5 million—$2.1 million less than 
what OntarioBuys had reported. Hospital 
management told us that they had not tracked 
any savings beyond the $2.5 million. When 
OntarioBuys gave us its revised list of savings 
subsequent to our audit, it indicated that the 
hospital had not included the annualized 
process savings in the report provided to us 
and that, with those included, the savings 
could be as much as $4.2 million. 

After the completion of our fieldwork, 
OntarioBuys gave us a revised list of cost savings 
that included savings from other projects that we 
were unable to confirm, having completed our audit 
some time earlier. However, the fact that reported 
savings for various projects changed dramatically 
after our audit raises questions about the reliability 
of the reported savings. 

For many projects, OntarioBuys reported 
savings that were determined without using 
baselines. As well, the definition of what constitutes 
“savings” varied among similar organizations. Our 
observations in this regard are the same as those 
made by numerous consultants to the Ministry and 
OntarioBuys. One pointed out that “it is simply 
impossible to prove benefits without baselines.” 
Another mentioned that “the lack of a proper 
definition of savings and a lack of instruction to 
the SSOs on how to account for savings” would 


“undermine the purpose for reporting savings.” Of 
even greater concern is that OntarioBuys did not 
objectively assess the validity of reported savings 
for most of the funded projects. 

Clearly, OntarioBuys must be more diligent in 
ensuring that its performance results are valid and 
supportable before reporting them. 


RECOMMENDATION 6 


To ensure that reported performance results are 
credible, OntarioBuys should: 

@ provide guidelines to shared-service 
organizations (SSOs) and broader-public- 
sector institutions on how savings are to 
be defined and how baselines are to be 
established and applied for the calculation of 
savings; and 

@ objectively assess and verify SSOs’ and 
projects’ reported savings to ensure that they 
are valid before publicly disclosing them as 
results achieved. 

OntarioBuys will be providing guidelines to 

shared-service organizations and BPS institu- 

tions on developing baselines required to 





calculate savings as part of the Supply Chain 
Guideline version 2.0 process already under- 
way. These guidelines will be provided by 
April 30, 2010. 

OntarioBuys will ensure that savings are 
verified prior to public disclosure and, where 
savings are projected, ensure they are identified 
as such. 


Other Performance Measures 


Guidelines 
In March 2008, the government directed 
OntarioBuys to develop a Supply Chain Guideline 
document to support and improve BPS supply-chain 
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activities. One year later, OntarioBuys published 
two principles to guide the BPS sectors: one was 

a code of ethics for supply-chain management 

and the other covered standards for procurement 
policy and procedures. At the time of our audit, 
OntarioBuys was developing additional principles 
for possible inclusion in an updated guideline docu- 
ment. After April 1, 2009, any transfer-payment 
agreement involving more than $10 million in 
annual funding from the Ministry of Health and 
Long-Term Care, the Ministry of Education, or the 
Ministry of Training, Colleges and Universities is to 
include the supply chain guidelines. 

The Ministry set OntarioBuys a performance 
target for the 2008/09 fiscal year with respect to 
the guidelines. OntarioBuys was to obtain endorse- 
ments from the Ontario Hospital Association and 
the Council of Academic Hospitals for its code of 
ethics and procurement guidelines. This perform- 
ance target had been achieved. 


Purchasing of Health Supplies 
A second performance target set by the Ministry 
for OntarioBuys to achieve in 2008/09 was to have 
50% of all medical, surgical, and consumable sup- 
plies addressed through SSOs. 
OntarioBuys told us that it had achieved this 
target as well. We questioned this assertion for the 
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following reasons: 

@ OntarioBuys did not track the actual 
purchases that flowed through the SSOs 
to determine whether the target was met. 
Instead, its reported achievement was based 
on the total purchases anticipated in the SSOs’ 
business cases, assuming that all SSOs have 
fully implemented their project plans. 

e The minutes of a meeting of SSO general 
managers on April 20, 2009, indicated that 
there was a need to increase the amount of 
group purchasing at hospitals because the 
current level was currently less than 25%. 
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Redirecting of Savings to Front-line Services 
In its 2004 Budget, the government stated that 
OntarioBuys was “an important initiative to reduce 
the overall costs of broader public sector...procure- 
ment and redirect savings to front-line services.” 
However, we found that OntarioBuys has no way of 
verifying whether savings are redirected to front- 
line services. Neither the performance measures 
of the Ministry’s results-based plan nor the supply- 
chain guidelines that OntarioBuys is developing 
require that the redirecting of savings to the front 
line be tracked. 


RECOMMENDATION 7 





To properly measure and report on performance 

results, OntarioBuys should: 

@ use actual purchase information from 
funded shared-service organizations (SSOs) 


to determine whether it has achieved the 
target percentage of having certain supplies 
purchased through them; and 

e develop performance measures and 
collect the information necessary to assess 
and report on the redirecting of savings 
generated by funded SSOs and projects to 
front-line services. 


OntarioBuys will use actual purchase informa- 
tion to determine whether the SSO participation 
metric has been achieved and will investigate 
developing the recommended performance 
measure(s) and collection of information 
regarding redirecting savings. 







Section 


Background 


The Ministry of Community and Social Services 
(Ministry) administers two acts under which it 
provides social assistance to approximately 450,000 
individuals as well as their qualifying family mem- 
bers for a total of more than 700,000 people. Under 
provisions of the Ontario Works Act, the Ministry 
provides employment and temporary income sup- 
port to some 200,000 individuals. This support is 
provided with the aim of helping recipients find and 
maintain paid employment. Under the Ontario Dis- 
ability Support Program Act (Act), the subject of this 
audit, the Ministry provides income and employ- 
ment support to approximately 250,000 individuals 
with eligible disabilities as defined by the Act. 
Eligible Ontario Disability Support Program 
(ODSP) disabilities include mental disabilities such 
as psychoses (for example, schizophrenia), neuro- 
ses (for example, depression), and developmental 
delays. Physical disabilities include diseases of the 
musculoskeletal system (for example, osteoarth- 
ritis), diseases of the nervous system (for example, 
Parkinson’s disease), and diseases of the circulatory 
system (for example, congenital heart disease). 
Although Ontario Works program income support 
is meant to be temporary, most ODSP recipients 
suffer from chronic disabilities and receive assist- 
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ance for many years. In some cases, they receive 
income support for the rest of their lives. 

To be eligible for ODSP income support: 

e all applicants must first demonstrate financial 
need by providing evidence that their liquid 
assets and income levels do not exceed speci- 
fied amounts; and 

e almost all applicants must be assessed to 
determine if their disability meets the eligibil- 
ity test established by the Act—no disability 
assessments are required for people who 
are receiving Canada Pension Plan disability 
benefits, for individuals aged 65 and over who 
are ineligible for Old Age Security, and for 
individuals residing in prescribed institutions 
such as psychiatric facilities. 

ODSP income support is intended to assist with 
basic living expenses such as food, shelter, clothing, 
and personal-needs items. Although employment- 
support programs are available to ODSP recipients, 
participation is not required. As a result, relatively 
few ODSP recipients join such programs. 

Income support provided to ODSP recipients 
is somewhat higher than that provided to Ontario 
Works recipients. A comparison of typical benefits, 
all of which are tax-free, between the time of our 
last audit in 2004 and the 2008/09 fiscal year is 
shown in Figure 1. 
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Figure 1: Typical Monthly ODSP Benefits (tax free) 


Source of data: Ministry of Community and Social Services 


Single Person 


basic needs allowance ($) 3 566 
maximum shelter allowance ($) . 454 
Maximum ODSP Benefit ($) 





1. child 12 years of age and under 


comparable Ontario Works benefit($) : 572 520 (gon 957 1,036 1,030 


Couple With One 
Spouse Disabled and 
One Child’ 


Single Person 


With One Child’ 





516 709? 7712 838? 
414 714 








2. reduction due to the introduction of the Ontario Child Benefit, up to $50 per month per child 


ODSP recipients may qualify for additional 
assistance, based on established need, for a number 
of other items, such as: 

e health-related necessities, including transpor- 
tation for medical appointments, medical sup- 
plies, special dietary items, and basic dental 
and vision care; and 

® community start-up and maintenance benefits 
to assist in the cost of establishing a perma- 
nent residence. 

ODSP is delivered by the Ministry’s 44 local 
offices under the supervision of nine regional 
offices. Although the cost of ODSP income support 
is shared between the province (80%) and the 
municipalities (20%), the municipalities’ portion 
will be reduced to 10% for the 2010 calendar year 
and eliminated in 2011 and beyond. In 2009, the 
province began paying 100% of the program’s 
administration costs. 

Largely as a result of caseload growth, as 
illustrated in Figure 2, total annual ODSP benefit 
payments have risen to more than $3 billion, a 42% 
increase since the time of our last audit in 2004. 

Since 2002, the Ministry’s information technol- 
ogy network, known as the Service Delivery Model 
Technology (SDMT) system, has supported the 
administration of both the Ontario Works and ODSP 
programs. SDMT, developed by a private-sector 
company at a cost of approximately $377 million, 
has been the subject of separate audits reported on 
in our 1998, 2000, and 2002 annual reports. 


Audit Objective and Scope 





Our audit objective was to assess whether the 
Ministry’s policies and procedures were adequate to 
ensure that: 

@ only eligible individuals received income sup- 
port and that the income support provided 
was timely and in the correct amount; and 

e the program was delivered with due regard 
for economy and efficiency. 

The scope of our audit included a review and 
analysis of relevant ministry files, policies, and 
procedures, as well as interviews with appropriate 
staff at the Ministry’s head office, three regional 
offices (Toronto, Central East, and Eastern), and 
the five local offices that we visited. We also held 
discussions and obtained information from a var- 
iety of organizations that are involved with, or have 
an interest in, the administration of the ODSP pro- 
gram, including two ODSP program client-advocate 
groups, as well as Legal Aid Ontario (which often 
represents applicants in their requests for benefits 
and in appeals before the Social Benefits Tribunal), 
and the Social Benefits Tribunal (which hears and 
rules on appeals regarding benefits that have been 
denied by the Ministry). 

Our work emphasized the policies and proced- 
ures in place with respect to the administration of 
the ODSP program during the 2008/09 fiscal year. 
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Figure 2: Annual ODSP Income-support Expenditures and Related Caseloads 


Source of data: Ministry of Community and Social Services 


Expenditures 





Fiscal Year 





2008/09 3.025 7.4 253,359 
2007/08 2.816 8.0 240,657 
2006/07 2.607 G2 228,885 
2005/06 2.433 5.8 215,628 
2004/05 2.299 1.9 208,070 
2003/04 eaten ~ 202,241 


We concentrated on areas with the largest program 
expenditures—basic needs and shelter assistance— 
which together constituted 97% ($2.93 billion) of 
total program expenditures. 

We reviewed the Ombudsman of Ontario’s 2006 
report on the Ministry’s Disability Adjudication Unit 
(DAU) that made a number of recommendations, 
including the need for timelier decision-making and 
the elimination of a four-month cap on retroactive 
benefit payments. We also considered the actions 
taken by the Ministry on these recommendations in 
planning our audit. 

We also reviewed several recent audit reports 
issued by the Ministry’s Internal Audit Services. 
However, the scope of those reports was generally 
limited to specific issues, as opposed to the payment 
of the basic needs and shelter allowance, which was 
the main focus of our audit. We were, therefore, 
unable to rely on those audits to reduce the scope of 
our work. 





Following our 2004 audit and the Ombudsman’s 
2006 report, the Ministry has taken steps to bet- 
ter administer the ODSP. For example, the hiring 


of additional medical adjudicators has allowed 
the Ministry to reduce the average wait time for 

a medical-disability decision to approximately 60 
business days, a significant improvement from the 


Caseload 


Average Expenditure Per Case 





ae 11,940 Oud 
5.1 11,700 2a 
6.1 11,389 1.0 
3.6 11,281 2.1 
29 11,049 4.9 

= 10,535 = 


time of our last audit. Another area of improvement 
since our last audit was the much better documen- 
tation in the disability adjudication files. 

Nevertheless, serious issues remain in determin- 
ing an applicant’s financial eligibility and the cor- 
rect amount of assistance to be paid. The Ministry 
has established a two-stage process to ensure that 
only qualified applicants receive income support. 
The first stage is problematic because it relies solely 
on the individual volunteering financial informa- 
tion. To compensate for the risks associated with 
this, the Ministry’s second stage requires third- 
party verification of certain information provided 
by the applicant. However, this verification require- 
ment is largely ignored in practice. 

As a result, the Ministry is not adequately 
ensuring that only eligible individuals receive 
disability support benefits and that the payments 
made to recipients are in the correct amount. Other 
significant findings and observations include the 
following: 

e Although the Ministry has significantly 
reduced the average wait time for a medical- 
disability determination decision, 60% of 
recipients sampled still received late pay- 
ments. On average, they experienced a 58-day 
delay after they had been determined to be 
medically qualified for payments, which is 
almost three times longer than the outside 
limit of 21 days established by the Ministry. 
These delays in receiving approved bene- 
fits offset to a significant degree the good 
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progress made since our last audit in expedit- 
ing the initial medical determination. 
Oversight procedures are lacking to monitor 
and assess the fairness and consistency of 
decisions made by individual adjudicators at 
the Ministry’s Disability Adjudication Unit 
(DAU). Consequently, eligibility determina- 
tion rates among adjudicators generally var- 
ied from 11% to 49%. 

Many initial decisions were overturned after 
applicants who were not approved for benefits 
by the Ministry appealed to the Social Benefits 
Tribunal. In fact, the Tribunal in the 2008/09 
fiscal year overruled the Ministry’s decisions 
in 55% of these appeals. An independent con- 
sultant hired by the Ministry in 2008 noted 
that many Tribunal members approved 100% 
of all appeals, while one member upheld all 
the Ministry’s decisions. 

Since 2002, the Ministry has not performed 
any of the periodic medical reassessments— 
required by legislation—to ensure continuing 
eligibility for disability support payments. As 
of March 31, 2009, there were 37,000 individ- 
uals identified as requiring such a reassess- 
ment to ensure that they still were eligible to 
continue receiving income support. Of those, 
11,000 were overdue, many by several years. 
The Ministry relies on one individual to do all 
the assessment and reassessment work for any 
given file, yet the individual’s work is neither 
supervised nor reviewed to ensure that the 
decisions made comply with ministry and 
legislative requirements. 

The total amount of overpayments for both 
active and inactive accounts has increased 
substantially to $663 million as of March 31, 
2009, from approximately $483 million at the 
time of our last audit in 2004. In many cases, 
overpayments resulted from what would 
appear to be recipients fraudulently misrepre- 
senting their circumstances. Often, these 
overpayments might have been avoided if the 
Ministry had followed up on tips received from 
the public, or more effectively reassessed the 


eligibility and the amounts to be paid to those 
individuals whom its own systems identified as 
high-risk. For example, for a number of years 
the Ministry ignored five complaints about one 
recipient family that was later established to 
have received more than $100,000 in overpay- 
ments. One of the tips noted that family mem- 
bers regularly drove new vehicles, including a 
new imported SUV. 

The Ministry’s computerized SDMT informa- 


tion system still lacks key internal controls, 
and regional and local offices are not receiv- 
ing, in an easily understandable format, the 
information they need to effectively oversee 
program expenditures. 





The Ministry of Community and Social Services 


welcomes the findings and recommendations of 
the Auditor General with respect to the delivery 
and oversight of the Ontario Disability Support 
Program. This is a vital service that ministry 
staff directly deliver to some of the most vulner- 
able citizens of Ontario. Within the context of 
a steadily growing caseload, the Ministry has 
taken numerous steps over the past several 

- years to improve customer service and business 
processes. Notwithstanding the gains made 
over the past few years, the Ministry recognizes 
the need for continuous quality improvement. 
In this regard, a number of initiatives that are 
focused on improved technical and business 
processes, customer service, and accountability 
have been introduced in the 2009/10 fiscal year. 


Detailed Audit Observations 





OVERVIEW OF PROGRAM DELIVERY 


From Figure 3, a schematic representation of the 
ODSP application process, it can be seen that 


Figure 3: ODSP Application Process 


Source of data: Ministry of Community and Social Services 
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disabled individuals in need of income support are 
normally referred to one of the Ministry’s 44 local 
offices to apply for ODSP benefits. If the individual’s 
financial need is considered immediate, he or she 
may be directed to the local Consolidated Muni- 
cipal Service Manager to apply for Ontario Works 
Assistance, which is generally granted more quickly 
than ODSP benefits. The individual can then apply 
to transfer to the longer-term ODSP program. If the 
individual’s need is not immediate, a caseworker 

in the local ministry office assesses the person’s 
financial eligibility for benefits through an income- 
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Application for Financial Assistance Approved 


and-asset test. To be financially eligible, a person’s 
total assets must be at or below: 

@ $5,000 for a single person; or 

@ $7,500, if there is a spouse in the benefit unit. 

(These amounts can increase by $500 for each 
eligible dependant.) 

Cash, bank accounts, RRSPs, and other assets 
that can be readily converted to cash are considered 
when calculating a person’s total assets. Certain 
assets, such as a principal residence, a primary 
vehicle, locked-in RRSPs, and trust funds in the 
amount of less than $100,000, are excluded when 
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determining whether the person’s assets are within 
the prescribed limits. 

When assessing a person’s income levels, a 
caseworker considers income from such sources as 
employment, the Canada Pension Plan, the Work- 
place Safety and Insurance Board, and Employment 
Insurance. Generally, to be eligible for even a 
partial ODSP benefit, 50% of the applicant’s total 
income from other sources must be less than the 
potential ODSP entitlement. 

Once an applicant’s financial eligibility has been 
established, he or she is provided with a disability- 
determination package. That package contains 
three forms: a health status and activities-of-daily- 
living index report; a consent form to have medical 
information disclosed to the ODSP; and an optional 
self-report. The first document, which must be 
completed by a physician or other prescribed health 
professional, provides information about the appli- 
cant’s medical condition(s) and impact on daily liv- 
ing activities. The consent form must be completed 
and signed by every applicant. Completing the 
third form, which is voluntary, gives applicants the 
opportunity to describe how their disability affects 
their daily life. 

The completed documents are forwarded to the 
Ministry’s centralized Disability Adjudication Unit 
(DAU) for review. An adjudicator, usually a profes- 
sional in the health-care field, reviews the forms and 
determines whether the individual meets the test for 
disability (as defined under the Act) and is, there- 
fore, entitled to assistance. If eligibility is approved, 
the DAU then advises the Ministry’s local office that 
referred the individual to commence benefit pay- 
ments. The Ministry’s target is that the first payment 
be issued within 21 calendar days of the disability 
determination. The amounts to be paid are now 
retroactive to the date the DAU received the com- 
pleted disability-determination package. 

If an adjudicator determines that an applicant 
fails to meet the test for a disability, the applicant 
may request an internal review. A team of three dif- 
ferent adjudicators reconsiders the application and 
must provide the reasons for its decision, in writing, 


to the applicant within 10 calendar days of receiv- 
ing the review request. 

An applicant whose claim is also rejected by 
the internal review team may appeal to the Social 
Benefits Tribunal within 30 calendar days of the 
internal review decision. 


INITIAL FINANCIAL ELIGIBILITY 
ASSESSMENT 


ODSP applicants must provide the Ministry’s 

local office with all the information necessary to 
establish their eligibility for income support and to 
determine the correct amount of eligible assistance. 
To do so, they must provide copies of a number of 
documents, most of which are to be visually verified 
by the intake worker and are intended to establish 
the identity and legal status of the individual. These 
include a social insurance number card, Ontario 
health insurance card, birth certificate, and docu- 
ments verifying a person’s status in Canada. Addi- 
tional documents, such as records verifying school 
attendance, may also be required, for example, for 
dependants over the age of 16. When it comes to 
disclosure of income and assets, the only require- 
ments are a representation by the applicant and a 
copy of a recent monthly bank statement. 

Our review of a sample of ODSP recipients’ files 
found that, although there were some instances 
where staff failed to review critical documents for 
establishing an applicant’s identity and legal status, 
this was adequately done in most cases. 

However, verifying an individual’s income and 
assets from personal representations and only one 
monthly bank statement is, in itself, not sufficient. 
For example, there is no assurance that an individ- 
ual has provided a bank statement for all of his or 
her accounts. Furthermore, an applicant could have 
withdrawn most of the money in the account before 
the bank issued the monthly statement. 

Accordingly, to help verify the income and assets 
declared by applicants, the Ministry has entered 
into a number of third-party, information-sharing 
agreements. Examples include arrangements with 


Human Resources and Skills Development Canada 
for employment insurance information, with the 
Canada Revenue Agency for tax return informa- 
tion, with Equifax for credit checks, and with the 
Family Responsibility Office to verify any support 
payments received. It is ministry policy that the 
completeness and accuracy of the declaration of an 
applicant’s income and assets must be verified with 
all four of these organizations. 

Nevertheless, we found that two of the three 
regional offices we visited did not verify an appli- 
cant’s income and assets information with any of the 
third-party providers, while the third office met the 
requirements only about one-fifth of the time. We 
concluded, therefore, that initial financial eligibility 
for ODSP recipients is not being adequately verified. 

We also note that just one ministry income- 
support specialist makes all the decisions with 
respect to assessing an applicant’s initial financial 
eligibility, and that the same individual maintains 
all the applicant’s case files. Supervisors are not 
required to conduct periodic supervisory reviews of 
decisions made and files maintained, and we saw 
no evidence that such reviews were ever under- 
taken. Such lack of oversight further increases the 
risk of payments to ineligible recipients. 


RECOMMENDATION 1 


To ensure that an individual’s initial financial 

eligibility for Ontario Disability Support Pro- 

gram benefits is adequately verified, the Min- 

istry of Community and Social Services should: 

@ comply in all cases with its own require- 
ments to verify an applicant’s declared 
income and assets with the third parties who 
have information-sharing agreements with 
the Ministry; and 

e conduct supervisory reviews, at least on a 
sample basis, of the decisions made and files 
maintained by intake caseworkers to ensure 
that staff are adhering to Ministry require- 
ments with respect to financial eligibility 
verification. 
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The Ministry recognizes the need to ensure 

that only those eligible to receive benefits 
through the Ontario Disability Support Program 
(ODSP) are admitted to the program. In order 
to improve customer service and program integ- 
rity, a new initiative, introduced in the 2009/10 
fiscal year, will streamline the ODSP case man- 
agement structure over the next two years. An 
integral part of this restructuring will be a clari- 
fication of accountabilities for all staff involved 
in the delivery of ODSP, including managers. 
Staff and managers will be supported to take on 
these clarified responsibilities through training 
and revised business processes. 


INITIAL DISABILITY DETERMINATION 


In our 2004 Annual Report, we expressed concern 
about delays in adjudicating applications for 
disability benefits and recommended that steps 
be taken to expedite this process. In 2006, the 
Ombudsman of Ontario conducted an investigation 
into the Ministry’s Disability Adjudication Unit 

to determine whether there were delays leading 
to applicants being deprived of benefits that they 
would otherwise be entitled to. The Ombudsman 
issued a report in May 2006 with a number of rec- 
ommendations, including the following: 

e@ The government of Ontario should amend 
the Act to eliminate the four-month limit on 
retroactive benefit payments. 

@ The Ministry should review its adjudication 
service standards and determine what the 
optimal processing time should be, given the 
intent and purpose of the program, and deter- 
mine appropriate staffing strategies to process 
applications expeditiously. 

e The Ministry should establish service goals for 
the treatment of pending applications. 

Subsequent to our audit in 2004 and the 

Ombudsman’s recommendations, the Ministry 
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eliminated the four-month restriction on retroactive 
benefit payments. As previously noted, eligible 
applicants’ retroactive benefit payments now cover 
the entire period from the time their completed 
application is received to the time when pay- 

ments commence, even if that period exceeds four 
months. 

For the year ending March 31, 2009, the DAU 
received approximately 34,000 new applications 
for benefits, 17% more than the 29,000 received in 
the year of our last audit in 2004. Over the same 
period, the number of adjudicators increased from 
30 in 2004 to 43 in 2009—a 40% increase. This has 
enabled the Ministry to reduce the average wait 
time for a medical-disability decision to approxi- 
mately 60 business days after the completed dis- 
ability-determination package is received. This time 
frame is well within the Ministry’s current internal 
goal of 90 business days to adjudicate all applica- 
tions. The average assessment period is a significant 
improvement from the wait times we found during 
our last audit in 2004 when many cases had not 
been adjudicated within four months. 

To help ensure that all applicants are treated 
fairly, they are assessed on a first-in/first-out basis. 
The Ministry has a triage process that requires that 
all new applications receive an initial review within 
10 business days of receipt. Approximately 25% of 
all such applications are determined, within the 
triage time frame, to have a clear, qualifying eligible 
disability. The remaining 75% of the applications 
require further review, and of these, approximately 
one-third are found to have an eligible disability. 
Our review of a sample of adjudication files noted 
that the reasons for assessment decisions were gen- 
erally well documented, which was often not the 
case in our 2004 audit. 

However, as a result of our review of the DAU’s 
adjudication process and the summary information 
provided us, we noted a number of concerns: 

e Responsibility for this disability determination 
rests with just one individual. In response to 
our 2006 follow-up report, the Ministry com-| 
mitted that its Chief Medical Adviser would 


annually review 50 randomly selected files 

from each of the unit’s adjudicators to ensure 

that correct medical assessments were being 
made. As of the completion of this audit, such 

a review had yet to occur. As a result, there 

still is no supervisory oversight or review pro- 

cess in place to assess the basis and quality of 
adjudication decisions. 

e@ The Ministry does not monitor the percent- 
age of applicants approved by individual 
adjudicators—either at the triage stage or 
subsequently. We found that these rates var- 
ied widely. For example: 

e At the triage stage, the percentage of 
applicants found to have an eligible dis- 
ability generally ranged from 13% to 45%, 
depending on the adjudicator. 

e The percentage of post-triage applicants 
found to have an eligible disability gener- 
ally ranged from 11% to 49%, depending 
on the adjudicator. 

The Ministry was unable to provide any explana- 
tion for these significant variances. The risk associ- 
ated with such wide disparities is that individuals 
with similar medical conditions can get a different 
decision, depending on which adjudicator’s desk 
their file lands on. 

Approximately two-thirds of applicants who are 
ultimately found to have no eligible disability ask 
for an internal review. These reviews are done by 
a team of three adjudicators. As a result of these 
reviews, some 15% of the original decisions are 
overturned, which, in our opinion, seemed reason- 
able. (However, an additional 55% of decisions that 
are further appealed are overturned by the Social 
Benefits Tribunal. See Social Benefits Tribunal 
Appeals later in this report.) 


RECOMMENDATION 2 


To ensure that all Ontario Disability Support 
Program applicants are adjudicated fairly and 
consistently, the Ministry of Community and 
Social Services shoula? 


@ periodically review a random sample of each 
adjudicator’s files to assess whether the deci- 
sions are generally supported and fair; and 

® monitor the percentage of applicants found 
to have an eligible disability by each adjudi- 
cator and, if there are significant variances, 
investigate the reasons for them and take 
corrective action where necessary. 


The Ministry has implemented several quality- 
assurance measures to ensure the integrity of 
decision-making (for example, team reviews). 
Further enhancements to existing quality- 
assurance processes will be introduced to ensure 
that clients applying for the Ontario Disability 
Support Program are consistently adjudicated. 
Regular file reviews by the Chief Medical 
Adviser are now under way. 


SOCIAL BENEFITS TRIBUNAL APPEALS 


Applicants who remain unsatisfied after an internal 
review can appeal to the Social Benefits Tribunal, 
an independent body that operates at arm’s length 
from the Ministry. The Tribunal hears two types of 
ODSP appeals: disability-determination decisions 
relating to an applicant’s eligibility for benefits and 
income-support decisions, which generally relate 
to disagreements over amounts to be paid and/or 
recovery of overpayments. 

In many cases, a Legal Aid Ontario lawyer 
represents applicants at the Tribunal. In fact, 
Legal Aid Ontario estimates that for its fiscal year 
ending March 31, 2009, its clinics represented 
approximately 7,500 ODSP cases that will cost 
approximately $15 million, almost one-quarter of 
its annual budget. 

For the year ending March 31, 2009, the out- 
comes of the Tribunal’s deliberations are outlined 
in Figure 4. 
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The average length of time between the request 
for an appeal and a tribunal member’s decision is 
approximately one year. In the fiscal year ending 
March 31, 2009, tribunal members overturned 
some 55% of ministry decisions, which is about 
10% lower than the corresponding average rate in 
the previous two years and 20% lower than at the 
time of our 2004 audit. However, given the signifi- 
cant increase in appeals, from some 2,700 in 2003 
to some 8,000 in 2009, the total number of appeals 
that resulted in a decision being overturned more 
than doubled to almost 4,400. 

The Ministry retained a consultant to investigate 
the reasons for the relatively high rates at which 
the Tribunal overturned its DAU decisions. The 
consultant identified a number of factors that may 
contribute to the relatively high overturn rate. The 
three most significant factors were: 

e The appellant appears in person at the tribu- 
nal hearing, but not during the DAU process, 
which is essentially a paper file review. 

e Although legal counsel often represent the 
appellant at the Tribunal, the Ministry’s 
six case-presenting officers appeared only 
about one-quarter of the time to explain the 
Ministry’s legal submission and rationale for 
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denying the initial appeal. 

e@ The Ministry and Tribunal use different 
criteria and processes for making decisions. 
For example, the DAU often denies benefits if 
evidence is conflicting, whereas the Tribunal 
seems to favour the appellant if the evidence 
is inconsistent or where ambiguity exists. 

In addition, the consultant noted that the 
Tribunal and the DAU appear to have a different 
interpretation of case law such as Gray vs. Director 
of ODSP (Ontario Court of Appeal, 2002), which 
broadened and liberalized the legal definition of a 
person with a disability under the Act. 

These issues notwithstanding, the consultant 
also noted that many tribunal members had an 
overturn rate of 100%, while one member upheld 
every DAU decision. 
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Figure 4: Social Benefits Tribunal Decisions, 2008/09 


Source of data: Social Benefits Tribunal 








Decisions Overturned 
Type of Appeal 






income support 
disability determination 


RECOMMENDATION 3 





To reduce the need for, and cost of, appeals 

and the relatively high rate at which the Social 
Benefits Tribunal overturns Ontario Disability 
Support Program eligibility decisions, the Min- 
istry of Community and Social Services should 
consult and work with the Tribunal to narrow 
the differences in approach to, and criteria used 
in, assessing individuals with a disability. In 
addition, to ensure that its rationale for denying 
a claim is clearly communicated to the Tribunal, 
the Ministry should ensure that it is represented 
by a case presenting officer at every hearing. 





The Ministry is in the process of analyzing the 


factors that may contribute to the Social Benefit 
Tribunal’s overturn rate of the Disability Adjudi- 
cation Unit’s decisions. On the basis of this 
review, the Ministry will identify potential legal, 
policy, or administrative strategies to address 
the issues. 

The Ministry is also in the process of 
reviewing how services can best be delivered, 
taking into account the bounds of existing 
resources. 


ELIGIBILITY REASSESSMENTS/ 
CONSOLIDATED VERIFICATION PROCESS 


Financial Eligibility Reassessments 


It is the Ministry’s policy that each local office 
should conduct a financial eligibility reassessment, 


Decisions Upheld 


Decisions Varied 


or Consolidated Verification Process (CVP), on 2% 
of its active caseload every month. In other words, 
approximately one-quarter of all active cases are 
to be reviewed every year. To help with the case- 
selection process, the Ministry’s head office pro- 
duces a number of ad hoc reports for local offices 
that identify potential high-risk recipients, either 
specifically or by profile. 

Although local offices generally met their 
monthly goal of conducting financial eligibility 
reassessments on 2% of active cases, the files 
reviewed were generally not selected from the high- 
risk group. Only one of the three regions we visited 
provided us with information on the review of high- 
risk cases and, in that region, just 15% of those cases 
were reviewed in the 2007/08 fiscal year. 

In addition, financial eligibility reassessments 
suffered from many of the same deficiencies 
previously noted for initial financial eligibility 
assessments. In most cases, the required third- 
party verifications of income and assets were not 
performed. The individual’s income and assets were 
determined solely on the basis of his or her declara- 
tion and from the review of just a single monthly 
bank statement, which cannot be relied on to give 
an accurate picture. We even noted that in some 
cases there were no new declarations obtained and 
no new bank statements reviewed. 

As noted later in this report, it is our view that 
if financial eligibility reassessments had been 
conducted on high-risk cases, and if the required 
procedures had been followed, there is the poten- 
tial for overpayments to be significantly reduced. 
For example: 

e A family of four started receiving $900 in 

monthly income support in 1996. Over the 


next two years, the Ministry received five 
complaints from five different people ques- 
tioning the family’s eligibility, including two 
complaints within the first two months of 
benefits being paid. One of the complainants 
noted that family members regularly drove 
new vehicles, including a new imported SUV. 
Although a financial eligibility reassessment 
was conducted on this recipient in 1999 
(after the five complaints were received), 

the reassessment did not investigate those 
complaints—and the benefits continued. 
Although a more in-depth investigation of this 
recipient’s financial eligibility commenced in 
2001, that investigation was not completed 
until 2005, at which time an overpayment 

of $104,000 was confirmed. The overpay- 
ment factors included undeclared income, 
undeclared assets, and cohabitation. At the 
time of our audit, the recipient was repaying 
this overpayment through $60-per-month 
deductions from current income support. 

A husband and wife started to receive $1,400 
in monthly income support in 1994. In 1999, 
the Ministry’s fraud hotline received a tip that 
they were ineligible, but this tip was never 
acted upon. In both 2001 and 2002, the recipi- 
ents were identified as high-risk and therefore 
deserving of a financial reassessment. How- 
ever, these reassessments did not take place. 
Finally, in 2004, a detailed financial eligibility 
reassessment found that the recipients had 
been living outside the country, not permitted 
under the Act, since 1998 and had received 
overpayments totalling $95,000. None of this 
Overpayment amount has been recovered. 

A recipient and dependent adult started to 
receive $1,040 in monthly income support in 
1991. In both 2001 and 2002, the recipient’s 
file was identified as in need of a financial 
eligibility reassessment, but no reassessment 
was undertaken during those years. Although 
a reassessment did take place in 2003, it did 
not result in any benefit adjustments. Another 


Ontario Disability Support Program Se 


financial eligibility reassessment in 2006 
found that the recipient had been receiving 
CPP disability payments since 1995, and that 
the amount of those payments made the 
recipient financially ineligible for income sup- 
port. As a result, an overpayment of $111,000 
was established, and benefits were terminated. 
None of this overpayment has been recovered. 
In most cases, including the preceding examples, 
the Ministry has had little success in getting recipi- 
ents to repay overpayments, nor are overpayments 
resulting from what would appear to be fraudulent 
representations referred to the police in most cases. 
We are concerned that individuals who receive 
ODSP income support under potentially fraudulent 
circumstances may have little reason to fear any 
consequences—even if they are caught. 


| RECOMMENDATION 4 


To ensure that recipients continue to be finan- 

cially eligible for Ontario Disability Support 

Program benefits and to avoid overpayments, 

the Ministry of Community and Social Services 

should: 

@ ensure that recipients identified as high-risk 
are prioritized for review; 
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@ comply in all cases with its own requirement 
to verify an applicant’s declared income and 
assets with the third parties with whom the 
Ministry has information-sharing agree- 
ments; and 

e be more proactive in following up on those 
tips that come from what appear to be bona 
fide sources. 


The Ministry agrees and has implemented in 
the 2009/10 fiscal year a revised risk-based 
approach to Ontario Disability Support Program 
(ODSP) financial eligibility reassessments. 
Future enhancements to this approach will 
include the development of a risk model for 
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social assistance that will help to ensure that 
only eligible recipients remain on the program. 

Also, to further support compliance with 
third-party-check and other eligibility review 
processes, the Ministry will be implementing a 
series of tools to reinforce its program manage- 
ment and oversight to ensure that all require- 
ments of ODSP service delivery are met. 

As noted above, the Ministry is introducing 
a new initiative that will streamline the ODSP 
case-management structure. An integral part 
of this restructuring will be a clarification of 
accountabilities for all staff involved in the 
delivery of the ODSP, including managers. 
Training and revised business processes will 
help to strengthen case-management activities 
related to verifying eligibility information 
received from outside parties. 


Management of Outstanding Tasks 


The Ministry’s Service Delivery Model Technology 
information system has a feature that allows the 
assignment of tasks and corresponding completion 
target dates to individual case files, as well as the 
tracking of outstanding tasks. A task is essentially a 
“to do” item that normally entails obtaining or veri- 
fying the information necessary to establish the con- 
tinued eligibility of a recipient and/or the correct 
amount of assistance. Tasks are system-generated 
for such things as recipients reaching the age of 60 
or 65 (at which time they may qualify for CPP or Old 
Age Security and receive income that could make 
them ineligible for ODSP benefits). Many of the 
remaining tasks are entered manually and are trig- 
gered by, for example, a complaint about a person’s 
eligibility or information obtained from third parties 
through the information-sharing agreements. 

It is essential that caseworkers review all 
outstanding tasks on a timely basis so that any 
necessary changes can be made promptly and over- 
payments can be avoided. 


Outstanding tasks have increased significantly 
since the time of our last audit. As of December 
2008, there were more than 206,000 outstanding 
tasks recorded in SDMT, 49,000 of which were 
overdue by a significant amount of time, with many 
overdue by more than five years. At the time of our 
2004 audit, there were some 57,400 outstanding 
tasks, excluding approximately 17,000 relating 
to overdue medical eligibility reassessments. The 
increase in outstanding tasks is particularly worri- 
some because it could affect an individual’s eligibil- 
ity for benefits, the likelihood of which is increased 
by the fact that there are no policies or procedures 
that require supervisory staff to review and monitor 
long-outstanding tasks. As a result, the information 
necessary to assess the eligibility of recipients and 
determine the correct amount of assistance may 
often not be obtained on a timely basis. 


RECOMMENDATION 5 





To ensure that Ontario Disability Support Pro- 
gram benefits are paid only to eligible individ- 
uals and in the correct amount, the Ministry of 
Community and Social Services should monitor 
case-management activities to ensure that tasks 
entered into its Service Delivery Model Tech- 
nology information system are followed up on 
promptly and that appropriate actions are taken 
to avoid overpayments. 





The Ministry agrees and has since simplified the 
technology relating to system-generated tasks. 


It is also reviewing current business processes 
for potential refinements or opportunities for 
improvements from a technology modernization 
perspective. 

As noted above, the Ministry is introducing 
a new initiative that will streamline the Ontario 
Disability Support Program (ODSP) case man- 
agement structure over the next two years. An 
integral part of this restructuring will be a clari- 
fication of accountabilities for all staff involved 


in the delivery of ODSP, including managers. 
This will include clarification of expectations 
with respect to the monitoring of overdue tasks. 


Medical Eligibility Reassessments 


An ODSP regulation and policy require that when a 
person is determined to have an eligible disability 
that may improve, a follow-up review must occur 
within two to five years. In the three-year period 
from 2006/07 to 2008/09, the DAU determined 
that just under half of the approved ODSP appli- 
cants, or approximately 24,000 individuals, had 

an eligible disability that might improve. All were, 
therefore, designated for review. 

However, the legislative requirement for medical 
reassessments notwithstanding, the Ministry has 
failed to conduct any such reassessment since 2002. 
In fact, during the 2005/06 fiscal year, the Ministry 
removed 34,000 recipients considered at low risk 
of improvement from the list requiring a medical 
reassessment but was unable to demonstrate how 
the low-risk determinations were made for those 
recipients. 

At the time of our audit, there were 37,000 
individuals who had been identified as requiring a 
medical reassessment to ensure that they were still 
eligible to continue receiving benefits. Of those, 
11,000 were overdue as of March 31, 2009, many 
by several years. 


RECOMMENDATION 6 


To comply with the Ontario Disability Support 
Program Act and to ensure that only eligible 
ODSP recipients continue to receive benefits, 
the Ministry of Community and Social Services 
should conduct the required medical reassess- 
ments within the legislated time frame. 


The Ministry is pleased to have introduced a 
new service standard with respect to the initial 
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adjudication of applications to the Ontario 
Disability Support Program (ODSP). While 

we have focused our resources and efforts on 
significantly improving initial adjudication 
timelines, the Ministry recognizes the import- 
ance of ensuring that only those individuals 
who have an ongoing need for ODSP continue to 
receive benefits. Therefore, the Ministry began 
conducting medical reviews in spring 2009. The 
Ministry will strive to conduct medical reviews 
within the required time frame. 


Income-support Payments to Individuals 


Given our concerns with respect to the effective 
implementation of both the initial and subsequent 
financial eligibility assessment processes, we 
reviewed a sample of payments and corresponding 
files. We noted the following: 

e@ Payments to individuals sampled commenced 
significantly later than the Ministry’s pre- 
scribed 21 calendar days after the DAU noti- 
fied an ODSP office that the applicant had an 
eligible disability and was, therefore, entitled 
to benefits. On average, payments to recipi- 
ents sampled commenced late almost 60% 
of the time, with the average delay being 58 
days. In many cases, there were delays of 
more than 100 days; one case was delayed 
195 days, or six-and-a-half months. These 
delays in receiving approved benefits offset 
to a significant degree the good progress 
made since our last audit in expediting the 
initial medical assessments. 

@ Ina few cases, information on file was 
incorrectly considered in determining the 
benefits entitlement, which resulted in 
either overpayments or underpayments. For 
example, the fact that one individual was 
incarcerated for 50 days and in receipt of CPP 
benefits was not considered correctly, resulting 
in overpayments of $4,200 over eight months. 
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Similarly, a caseworker failed to consider a 
recipient’s written confirmation and sup- 
porting documentation that his orphan bene- 
fits under CPP had been discontinued, which 
resulted in an ODSP underpayment of $200 
per month, or $1,000 at the time of our audit. 

Although the individual amounts of overpay- 
ments in our sample were generally small, collect- 
ively they may well be significant. 

We also noted that, province-wide, the total 
spent on special dietary allowances has increased 
substantially since the time of our last audit in 
2004. At that time, the payments totalled $18.1 mil- 
lion; in the 2008/09 fiscal year, the amount 
exceeded $104 million, more than a five-and-a-half 
fold increase. We found that many payments for 
special dietary allowances to purchase particular 
foods, which must be authorized by an approved 
health professional, seemed questionable. For 
example, for one family of 10, all 10 people received 
a monthly special dietary allowance totalling 
$2,475 per month, resulting in total monthly ODSP 
assistance of $4,163, or nearly $50,000 per year, 
tax-free. Another example concerned a family of 
nine, where all nine received a monthly special diet- 
ary allowance totalling $2,194 per month, resulting 
in total monthly income support of $4,540, or nearly 
$55,000 per year, tax-free. In addition, we found 
that, in some cases, payments for supplementary 
benefits such as community start-up and special 
dietary allowances were made without the required 
documentation and receipts being on file. 


RECOMMENDATION 7 





To ensure that eligible applicants receive the 
correct financial entitlements within a reason- 
able time frame, the Ministry of Community and 
Social Services should ensure that: 

@ Ontario Disability Support Program pay- 
ments start within the prescribed 21 
calendar days of the determination that the 
person has an eligible disability; 


e allofthe information necessary to determine 
the correct amount of benefits is on file and 
correctly considered before payments are 
made; and 

@ suspicious or unusual circumstances, includ- 
ing those relating to the special dietary 
allowance, are appropriately flagged for 
additional follow-up. 





The Ministry agrees and is taking steps to ensure 
that eligible applicants receive the correct finan- 


cial entitlements within a reasonable time frame. 
The Ministry will be implementing a series 

of tools to reinforce its program management 

and oversight to ensure that all requirements 

of Ontario Disability Support Program service 

delivery are met. 


OVERPAYMENTS 
Determination 


Overpayments occur when recipients are paid 
more assistance than they are entitled to receive. 
Information contained in the Ministry’s com- 
puterized SDMT tracking system indicates that 
outstanding overpayments have increased sub- 
stantially since the time of our last audit in 2004. 
The total estimated overpayments now stand at 
$663 million, compared to $483 million in our 
earlier audit. Figure 5 shows the increase in over- 
payment amounts, and Figure 6 shows the number 
of cases involved. 

The reason for any overpayment and how the 
overpayment was calculated should be properly 
documented in either SDMT or the recipient’s 
paper file. In practice, this is often not the case. 
For example, with respect to information in SDMT, 
we noted the following: 

e The reason for an overpayment is often too 

general; for example “eligibility change” is 


often cited, which makes it difficult to attrib- 
ute the overpayment to specific changes in 
circumstances. 

e@ The SDMT often does not show how the 
overpayment was arrived at, making it impos- 
sible for the caseworker to determine how the 
amount was calculated so that an explanation 
can be provided to the recipient. 

e Ifthe caseworker makes multiple changes of 
information in SDMT at once, each of which 
results in an overpayment, the system often 
does not record all of the overpayments. 

e The system cannot calculate overpayments 
incurred prior to 2002, the year SDMT was 
introduced. 

Given the above lack of detail in SDMT, case- 

workers in most instances do not attempt to verify 





Figure 5: Total Overpayments, 2003/04-2008/09 


($ million) 
Source of data: Ministry of Community and Social Services 


Ma active cases 
[1] inactive cases 





* Amounts are as of December 31, 2003, as reported in our 
2004 Annual Report. 
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the completeness and accuracy of the overpayment 
information recorded in SDMT. 


Collection 


ODSP benefits are subject to deductions to recover 
any prior overpayments under the Ontario Works 
Act as well as the Ontario Disability Support Pro- 
gram Act. Current ODSP benefit deductions can also 
be used to recover prior overpayments under these 
acts’ predecessors, the General Welfare Assistance 
Act and the Family Benefits Act. In all cases, overpay- 
ments are calculated without interest. 


Overpayment Recovery—Active Accounts 
Overpayments on active accounts are recovered 
primarily through automatic deductions from the 
recipient’s monthly income-support allowance. 
The maximum allowable monthly deduction is 
10% of the recipient’s combined basic needs and 
shelter allowance. In practice, however, the Min- 
istry imposes just a 5% repayment rate, half the 
legislated maximum. The monthly deduction can 
be further reduced or eliminated entirely should 
the Ministry determine that a 5% benefit reduc- 
tion would cause the recipient undue hardship. 
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Although the Ministry was unable to provide us 
with the number of active overpayment cases for 
which no recoveries are being made, we found 
among the files we reviewed that about one in five 
overpayment accounts was being exempted from 
deductions. 

The amounts of overpayments being collected 
from active cases through automatic deductions of 
current benefits are shown in Figure 7. 





Figure 6: Overpayment Cases, 2003/04-2008/09 


Source of data: Ministry of Community and Social Services 
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Figure 7: Overpayments (written-off and collected) 
Source of data: Ministry of Community and Social Services 





2003/04 





Amounts Collected Vee 
active cases ($ million) 31.6 32.6 





inactive cases ($ million) 16.2 10.6 


Written-off ; sete 
amount ($ million) — yet 
# of cases involved — 5,984 


Our review of a sample of active accounts in 
which overpayments were being recovered from 
current benefits found that payment amounts were 
generally small in relation to the amount of over- 
payment. As a result, recovering even a small por- 
tion of the total overpayments will take many years 
and full recovery is unlikely. For example: 

@ One individual owing $78,000 had his current 
benefits reduced by $10 per month. If the 
recipient made no other form of repayment 
and the recovery rate remained the same, it 
would take approximately 650 years to collect 
the outstanding amount. 

e Benefits of another individual with a $102,000 
overpayment were being reduced by $58 per 
month. If the recipient made no other form of 
repayment and the recovery rate remained the 
same, it would take approximately 147 years to 
collect the outstanding amount. 

Given many recipients’ general inability to 

repay overpayments, it is all the more important to 
strengthen internal controls and avoid such over- 
payments in the first place. 


Overpayment Recovery—Inactive/Terminated 
Accounts 
As was the case at the time of our last audit in 2004 
the Ministry’s initial collection effort for inactive/ 


*) 


terminated accounts consists of sending three “dun- 
ning letters” (debt notices) over a 60-day period, 
requesting that the debtor arrange a plan to repay 
the outstanding amount. If there is no response 


PAV VA 2006/ 
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within 30 days of the third letter being sent, it has 
been the Ministry’s practice since 2005 to transfer 
the account to its internal Overpayment Recovery 
Unit (ORU)— in effect, an in-house collection 
agency. 

The ORU sends an additional two dunning 
notices. The first advises the recipient that the 
recovery unit has been assigned responsibility for 
the debt and unless arrangements are made within 
30 days to pay the outstanding amount, the Min- 
istry will garnish any future tax refunds from the 
Canada Revenue Agency. Before taking this step, 
the unit sends a second and final letter, providing 
another 15 days to settle the amount owing. 

Our review of a sample of overpayment files 
found the following: 

e About 40% of overpayment cases had not 
been transferred to the ORU as required, even 
though the overpayment had been known, 
on average, for about three years. We were 
advised that, in most cases, the reason for not 
transferring the file was a lack of resources to 
do the necessary paperwork. 

e About 50% of eligible files were transferred 
to the ORU between a year or two after the 
overpayment was identified. 

e About 10% of the accounts were ineligible for 
transfer to the ORU for a variety of reasons, 
which in many cases included a pending 
appeal to the Social Benefits Tribunal of the 
amount outstanding. 

Summary information provided to us by the 

ORU indicated that in the period from the unit’s 


inception in October 2004 to March 31, 2009, 

the unit received and is currently administering 
approximately 23,000 overpayment accounts from 
inactive or terminated accounts with a total value 
of $141.8 million. (This does not include approxi- 
mately 28,000 accounts totalling $42 million that 
was written off during that time.) With respect to 
these 23,000 accounts, we found the following: 

e About 3,200 accounts totalling $12.4 million 
have not been subject to any collection effort, 
many for more than two years. 

e About 5,300 accounts totalling $40.2 million 
have been subject to collection efforts, but 
with no success to the time of our audit. 

@ About 14,500 accounts with an outstanding 
balance of $89.2 million have been either 
referred to the CRA or have entered into 
voluntary repayment plans for which some 
amount has been collected. However, the 
amounts collected from 6,300 accounts since 
the inception of the unit total just $7.6 mil- 
lion, or approximately 5% of the original 
$141.8 million in identified overpayments 
transferred to the unit since 2004. 

@ Overpayments forwarded to the ORU are not 
reviewed or assessed with respect to the indi- 
vidual’s ability to pay. As a result, the Ministry 
is unable to take advantage of the opportunity 
to focus on the recovery of amounts from 
former recipients who, for example, have 
returned to work or acquired liquid assets of 
considerable value and, therefore, have the 
means to pay. 


Temporarily Uncollectible Overpayments 
As of March 31, 2009, the Ministry has designated 
$59 million of overpayments as “temporarily 
uncollectible.” Of this amount, approximately 
$43 million relates to overpayments transferred 
from predecessor programs. These monies are 
considered uncollectible largely because the over- 
payments are poorly supported and/or have been 
outstanding for prolonged periods of time ranging 
as high as 16 years. 
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Most of the remaining amounts are designated 
temporarily uncollectible for several reasons, 
including hardship on the recipients or because the 
Ministry cannot substantiate the amount overpaid. 

Although the Ministry advised us at the time of 
our last audit in 2004 that it intended to establish 
the validity and collectibility of all the then- 
outstanding overpayments by December 2005, this 
has not occurred. Given that little has been done 
for so many years to collect this money and that 
most of the overpayments were made years ago, we 
doubt whether the Ministry will ever collect much 
of the $59 million. If this is indeed the case, these 
accounts should be written off. 


RECOMMENDATION 8 


To better utilize its limited resources and help 
maximize the recovery of Ontario Disability 
Support Program overpayments, the Ministry of 
Community and Social Services should: 

e devote more efforts to minimize overpay- 
ments in the first place, given the limitations 
in recovering significant overpayments from 
active and inactive recipients; 

@® ensure that overpayments from inactive 
accounts are transferred to, and followed up 
on by, the Ministry’s Overpayment Recovery 
Unit on a timelier basis, with emphasis on 
accounts that are considered to have the 
most potential for repayment; and 

@ assess the validity and collectibility of out- 
standing overpayments designated as tem- 
porarily uncollectible and, where warranted, 
recommend that they be written off so that 
attention can be focused on those accounts 
where collection efforts are likelier to yield 
results. 


The Ministry agrees that overpayment recovery 
must be maximized and has recently imple- 
mented business and technology changes to 
improve the recovery of overpayments. 
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To minimize the possibility of overpayment, 
the Ministry recently introduced changes to the 
way that Ontario Disability Support Program 
(ODSP) recipients report income. The Min- 
istry will continue with its efforts to mitigate 
overpayments and has recently implemented a 
risk-based approach to ODSP financial eligibility 
reassessments. Future enhancements to this 
approach will include the development of a 
risk model for social assistance that will help to 
ensure that only eligible recipients remain on 
the program and that the payments they receive 
are accurate. 

Finally, the Ministry is assessing the feasibil- 
ity of accelerating the write-off of aged overpay- 
ments and prioritizing overpayment collection 
on the basis of recipients’ and former recipients’ 
ability to repay. 


CASE MANAGEMENT 
Workload 


Our previous audit commented on the then 
relatively high number of files per caseworker. 
Subsequent to our 2004 audit, the Ministry 

hired additional caseworkers. Figure 8 shows the 
resulting reduction in average caseloads compared 
to the time of our last audit. 

Clearly, the average caseload has decreased 
significantly from the time of our last audit, by 
about 35% overall. However, caseloads still varied 
significantly among the Ministry’s nine regional as 
well as among its 44 local offices, with some offices 
having double the caseload per caseworker than 
others. Despite the overall decrease in caseloads, 
there is little evidence that the quality of work 
has improved, especially in conducting financial 
eligibility assessments and in clearing outstanding 
tasks. Our review of a sample of files continues to 
show many lingering problems, including, as noted 
above, a significant increase in overpayments since 
our last audit. 


Figure 8: Caseload Comparison, December 2003 and 
March 2009 


Source of data: Ministry of Community and Social Services 


December March % 





2003 2009 Decrease 





-RegionalAverages 


highest 465 318 32 
lowest 340 230 32 
Local OfficeNversges 
highest not available 351 
lowest not available 161 


Furthermore, the Ministry has no standards 
in place to assess whether staffing is sufficient to 
adequately perform all necessary case-management 
functions and to ensure that the ODSP program is 
well administered. 

We also noted that caseworkers in the three 
regions we visited took, on average, more than 20 
sick days per year, which compares unfavourably to 
the overall Ontario Public Service average of about 
10 days per year. 





"RECOMMENDATION 9 





To ensure that Ontario Disability Support Pro- 
gram caseworkers can effectively carry out their 
responsibilities, the Ministry of Community and 
Social Services should: 

@ assess caseworkers’ responsibilities and work 
processes to establish reasonable caseload 
benchmarks in each of the 44 local offices; 
and 

@ strengthen efforts to monitor sick leave and 
set targets for reducing absenteeism to more 
reasonable levels. 





The Ministry continues to review how services 


can best be delivered within the bounds of 
existing resources. Building on business and 
technical improvements, to assist with case 


management, the Ministry is planning to imple- 
ment a new Ontario Disability Support Program 
(ODSP) service delivery model that will simplify 
the way ODSP recipients are supported and 
address staff workload issues. At the same time, 
the Ministry will strengthen measures to mon- 
itor sick-leave usage and ensure that the Ontario 
Public Service Attendance Support Program is 
applied appropriately province-wide. 


Service Delivery Model Technology System 


The Ministry’s Service Delivery Model Technol- 
ogy system, the primary information technology 
network to support social assistance delivered by 
both Ontario Works and ODSP, was implemented 
province-wide in early 2002. It was developed to 
provide a common database with real-time access 
to case information and to reduce administrative 
costs while freeing up caseworker time to allow for 


better customer service to applicants and recipients. 


However, as was the case at the time of our pre- 
vious audits—of Ontario Works in 2002 and ODSP 
in 2004—we found that caseworkers still expressed 
considerable dissatisfaction with the SDMT system. 

Concerns expressed included the following: 

@ SDMT continues to make errors that ministry 

staff cannot explain. For example, although 
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the system is supposed to automatically 
detect all Ontario Works payments made to a 
recipient during any period for which ODSP 
benefits are granted retroactively, automatic 
deductions for duplicate Ontario Works 
payments were often not made, resulting in 
overpayments. 

e Regional and local offices are not receiv- 
ing, in an easily understandable format, the 
information they need to effectively oversee 
program expenditures. For example, some 
pre-programmed reports are incomplete and 
inaccurate. As a result, ministry staff have 
created manual systems or workarounds for 
tracking functions such as intake, internal 
reviews, and tribunal appeals. 

In addition, the system lacks certain basic inter- 
nal controls. For example, frontline caseworkers 
have the ability to create a client file, initiate and 
approve payments, and close files without super- 
visory review and approval. In effect, they have 
considerable powers to act without management’s 
knowledge. 

A more complete discussion of the issues and 
concerns with respect to the SDMT system is 
included in this chapter’s VFM Section 3.11 on the 
Ministry’s Ontario Works program. 


PT | 
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Section 





Background 


The Ontario Research Fund (Fund) program was 
created in 2004 to “support scientific excellence 
by supporting research that can be developed 
into innovative goods and services that will boost 
Ontario’s economy.” The program aims to keep 
Ontario’s researchers at the leading edge by sup- 
porting the direct and indirect operational costs of 
research through its Research Excellence Program, 
and the capital costs of research through its Large 
Infrastructure Program and Small Infrastructure 
Program. The Fund is the responsibility of the 
Ministry of Research and Innovation (Ministry), 
which was created in 2005 and focuses its efforts 
on activities that support Ontario’s knowledge 
economy and create high-value jobs. Previously, 
research funding had been delivered by the Min- 
istry of Economic Development and Trade under 
various other programs. The guiding principles of 
the new ministry, as stated in the Ontario Innova- 
tion Agenda, include extracting value from public 
research investments through commercialization 
and investment in research that will create jobs. 
The Ministry has dedicated approximately 15 
full-time employees and five support staff to deliver 
the Fund program. There is no specific legislation 
related to the Fund, as the program was established 


iF ]eCymee Ministry of Research and Innovation 


ei] Ontario Research Fund 


by an approved cabinet submission. The program 
provides research grants to institutions, primarily 
universities, and requires the research institution 
to obtain private-sector and institutional support 
in addition to program funding. The program also 
requires an advisory board and peer review panels 
to review research proposals and recommend pro- 
jects for funding. 

Since the inception of the Fund in 2004 to 
March 31, 2009, transfer payment expenses for 
the capital and operating components of the pro- 
gram have amounted to $303 million, with total 
announced program commitments of $623 mil- 
lion, as shown in Figure 1. Other commitments 
as reflected in Figure 1 represent contribution 
commitments from research institutions and the 
private sector. 


Audit Objective and Scope 


The objective of our audit of the Ontario Research 





Fund was to assess whether the Ministry had satis- 
factory systems and procedures in place to: 
@ measure and report on the program’s effect- 
iveness in fulfilling its objectives; 
@ ensure that resources were being managed 
with due regard for economy and efficiency; 
and 
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Figure 1: Ontario Research Fund Program Commitments, 2004/05-2008/09 


Source of data: Ministry of Research and Innovation 


# of 


Program Projects 


nce Program (operating) _ 






Large Infrastructure Program 89 
Small Infrastructure Program 906 








@ ensure compliance with government direc- 


tives, ministry policies, and contractual 
arrangements. 

The scope of our audit included discussions with 
ministry staff, an analysis of relevant files and other 
documents, and a review of research programs 
and practices in other jurisdictions. In addition, we 
spoke with several peer review panel members who 
review Research Excellence Program proposals for 
their input on the program. Our audit also included 
a review of the activities of the Ministry’s Internal 
Audit Services Branch. We reviewed the Branch’s 
recent reports and incorporated any relevant issues 


into our audit work. 





In our 2003 audit of the Science and Technology 
Division of the former Ministry of Enterprise, 
Opportunity and Innovation, we reported signifi- 
cant concern over the lack of effective governance 
and accountability mechanisms in place for the 
management of various Ontario research programs. 
At the time, the province was using external 
agencies to administer key government research 
programs. The consolidation of operating and 
capital research funding into one comprehensive 
program that is fully managed and administered by 
the Ministry of Research and Innovation (Ministry) 
has helped to address this concern. The Ministry 
has also established a fair and transparent peer 


($ million) 






Total Ministry Other Total 
Expended Commitments Commitments Commitments 


($ million) ($ million) ($ million) 


117 ae 519 692 
105 144 233 377 






review process to evaluate research proprosals and 
make funding recommendations to the Minister. 
However, there are still a number of areas where 
improvements are required. For example: 

@ The Ontario Research Fund’s (Fund’s) overall 
mandate emphasizes supporting research 
that will provide economic and social benefits 
for the people of Ontario through the com- 
mercialization of research from our publicly 
funded research institutions. In a 2004 
appearance before the Standing Committee 
on Public Accounts, the then Deputy Minister 
stated, “we need to heighten the focus on 
results, commercialization, that far end of the 
spectrum.” However, $623 million has been 
committed to research projects in the province 
and most of the research funded was for basic 
(that is, theoretical) research that was not 
focused on commercial potential. 

@ In our 2003 audit, we noted that the Ministry 
did not have adequate guidelines in place 
requiring the institutions it funds to ensure 
that the research funded will ultimately 
benefit Ontario. The current policy is to 
delegate intellectual property management 
to the institution and allow ownership of any 
benefits to vest with either the institution or 
the researcher, who could move to another 
jurisdiction. This practice has resulted in intel- 
lectual property policies that are inconsistent 
from one institution to another. We noted that, 
to ensure that research benefits remain within 
the funding jurisdiction, other jurisdictions 
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require ownership of intellectual property to 
rest with the research institution. 

The Ministry now reports on how its programs 
are performing against three stated targets: 
the dollar value of investments leveraged from 
the private sector, the number of individuals 
with enhanced skills involved in Ministry- 
funded projects, and active licences that 
resulted from Ministry-funded projects. How- 
ever, the Ministry does not measure or report 
publicly on the program’s contribution to the 
Ministry’s overall strategy of creating high- 
paying jobs and commercializing research. 
More information is required to assess whether 
the program is achieving its objectives. 

The Ministry did not have an adequate process 
in place to ensure that the projects funded 
through the Large Infrastructure Program 
supported Ontario’s strategic priorities or 
provided strategic benefits to Ontario. The 
Ministry generally based its funding on the 
decisions of the Canada Foundation for Innov- 
ation (CFI), with the result that the province 
funded projects worth $41.5 million that 

did not directly support Ontario’s strategic 
priorities. In addition, although we found that 
the Research Excellence Program selection 
process was fair and in accordance with pro- 
gram policies, $65 million of program funding 
was allocated to some very large projects 
where it was questionable whether they met 
the program’s eligibility criteria, although we 
were advised that they were very worthwhile 
projects for Ontario. 

The Ministry relied on federal CFI processes 
to monitor Research Infrastructure Program 
grants and did not sufficiently assess or review 
the CFI’s work to ensure that more than 

$300 million in program funding commit- 
ments was being spent for the approved pur- 
pose. The Ministry does not perform its own 
site visits because it is entitled to receive the 
results of CFI site visits and audits of Ontario 
co-funded projects. However, we found that 


the Ministry had not requested or received 
any of this information from the CFI. 

e@ Ontario’s colleges tend to focus on applied 
programs and research and helping small- to 
medium-sized businesses develop new or 
improved technologies and processes for the 
marketplace. Such research provides direct 
opportunities to contribute to Ontario’s 
economic growth. Colleges are eligible for 
funding through the Fund, but since the 
inception of the program no funding has been 
awarded directly to Ontario’s colleges. Given 
that commercialization of research projects 
is one of the key program objectives, the 
Ministry should assess the potential benefits 
of applied research projects that both address 
the unique needs of Ontario’s colleges and 
offer enhanced commercialization potential. 

e As part of the monitoring process for the 
Research Excellence Program, the Ministry 
receives various reports from grant recipients. 
However, we found that the Ministry had not 
performed any formal monitoring or clarified 
its expectations for independent audits to ver- 
ify the information submitted by recipients, to 
determine whether program funds were being 
spent for the intended purpose, and to gauge 
whether the recipients’ performance was 
satisfactory. 





Detailed Audit Observations 


PROGRAM OBJECTIVES, BENEFITS, AND 
REPORTING EFFECTIVENESS 


Program Objectives 


The report entitled The State of the Nation 2008: 
Canada’s Science, Technology and Innovation System 
issued by the Science, Technology and Innovation 
Council concluded that, despite Canada’s signifi- 
cant strengths in many fields of research, it was 
not translating its strength in basic science into 


sustained commercial success as effectively as other 
nations have done. 

The Ontario Research Fund (Fund) program was 
created under the Ministry of Economic Develop- 
ment and Trade’s 2004 science and technology 
strategy, which is still the guiding document for the 
_ program. This strategy was developed to support 
government priorities and to help Ontario achieve 
long-term prosperity through innovation that cre- 
ates high-paying jobs, provides people with the 
skills they need for those jobs, and brings leading 
products to market. Specific goals of the strategy 
are to improve Ontario’s performance in commer- 
cializing research, sustain research excellence, and 
leverage funding from private-sector partners. 

When the government introduced the new 
research program in the Legislature, the Minister 
made the following statement: 


We're creating a new Ontario Research 
Fund, which will do three things: It’ll 
make us more accountable and transpar- 
ent, make certain that there is a made- 
in-Ontario set of policies toward research 
and commercialization, and place a 
greater emphasis on commercialization as 
well. In addition to this, we have a com- 
mercialization strategy which will take 
good ideas out of our labs and ensure that 
they get to the marketplace with some 
degree of success. 


The Cabinet submission proposing the Fund also 
emphasized the importance of commercialization 
of funded research. 

In 2005, responsibility for the Fund shifted 
to the newly created Ministry of Research and 
Innovation (Ministry). The guiding principles of 
the new ministry, as stated in the Ontario Innova- 
tion Agenda, include extracting value from public 
research investments through commercialization 
and investing in research that will create jobs, 
contribute to a cleaner environment, and result in 
better health care for Ontario families. 
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The Fund contributes to Ontario’s science and 
technology strategy and the Ministry’s overall man- 
date by supporting research that can be developed 
into innovative goods and services to advance 
Ontario’s economy. Specifically, one of the goals 
of the Research Excellence Program is to focus on 
commercialization of research, while the Research 
Infrastructure Program is meant to ensure that 
institutions have state-of-the-art infrastructure to 
engage in technology development. 

The importance of commercialization of 
research to Ontario was noted in the original cab- 
inet submission, but we found that most funding 
was for basic theoretical research as opposed to 
applied research, which is more focused on com- 
mercial potential. With the Research Excellence 
Program, the Ministry does not formally keep track 
of, or report on, the percentage of projects that 
have applied or commercial value. For the Large 
Infrastructure Program, we found that almost 80% 
of the funds requested in 2007/08 were for basic 
research (that is, lacking commercial potential), 
and, consequently, were not aligned with commer- 
cialization, one of the program’s goals. 


| RECOMMENDATION 1 


To ensure that the Ontario Research Fund 
(Fund) program supports the Ministry of 
Research and Innovation’s (Ministry’s) overall 
strategy of job creation and is consistent with the 
Fund’s commercialization objective, the Ministry 
should place more emphasis on funding projects 
that have viable commercial potential. 


The Ministry will ensure that commercialization 
potential continues to be evaluated during the 
selection process and that commercialization 





milestones are met as Fund projects progress. 
Commercialization potential is a key criterion 
of any funding decision of the Fund, together 
with research excellence, strategic value, and 
relevance to Ontario. 


Chapter 3 ¢ VFM Section 3.10 
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Benefits of Research Projects 


The benefits of research investments should make 
an important contribution to Ontario’s prosper- 
ity. Research produces ideas and knowledge that 
stimulate economic and social growth with new 
and improved technologies and products, and the 
creation of new companies and industries. Research 
discoveries achieve commercial value when they 
are put to use by industries and businesses. The 
Fund program is intended to support research 

that can be developed into innovative goods and 
services that will boost Ontario’s economy; con- 
sequently, ensuring commercialization of research 
results is important for Ontario to realize the bene- 
fits of publicly sponsored research. 

Commercialization of research results developed 
within an institution is typically accomplished 
through the transfer of intellectual property (IP) to 
an existing or new company. Intellectual property 
rights represent the legal ownership resulting from 
research and academic activities that can result in 
patents, trademarks, and copyrights. The owner of 
intellectual property has the right to exclude others 
from using it, and ownership can be transferred or 
sold. 

The Ministry has cited several concerns regard- 
ing intellectual property in Ontario, including: 

e@ Businesses are often unaware of the intellec- 
tual property created within public research 
institutions. 

e@ Industry’s access to intellectual property is 
hindered by a lack of consistent policies across 
Ontario research institutions. 

e The commercialization of intellectual 
property may require complex and time- 
consuming negotiations, often with several 
institutions with different policies. 

In an appearance before the Standing Com- 
mittee on Public Accounts in 2004, the then 
Deputy Minister stated: “The approach taken in 
the development of intellectual policy options will 
be consistent with similar programs in competitor 
jurisdictions in Canada and North America. The 
target date for completion is the end of this year.” 


In a 2006 report to the Premier, the Ontario 
Research and Innovation Council stated that 
“Ontario needs an effective intellectual property 
system that ensures a healthy expedient flow of 
intellectual property out of universities to mar- 
ket,” and recommended that “the Government of 
Ontario should also ensure that IP access policy is 
enshrined within public sector funding agreements, 
in order to promote knowledge and IP transfer of 
publicly funded research.” 

We reviewed the Fund program guidelines and 
other ministry documents to determine what the 
Ministry does to ensure that potential research 
discoveries benefit Ontario and its taxpayers. We 
found that the Ministry “delegates intellectual 
property management to institutions on the expect- 
ation that it will be managed in the best interest of 
Ontario, and encourages research institutions to 
use best practices in managing IP and transferring 
technology to the marketplace.” However, the Min- 
istry did not advocate or have general guidelines or 
best practices in place to assist research institutions 
in identifying, protecting, and commercializing 
intellectual property to maximize the benefits of 
research for Ontario. 

Program guidelines indicate that the Ministry 
does not claim ownership or rights to any intellectual 
property resulting from research funded through 
the program, and that these rights are determined 
by the institutions’ intellectual property policy. The 
Ministry’s contracts under the Research Excellence 
Program allow intellectual property rights to belong 
to either the researcher or the recipient institution, 
while contracts under the Research Infrastructure 
Program contain no clauses regarding intellectual 
property rights. 

The Ministry informed us that, as a condition of 
funding, program proposals require a description of 
the institution’s policy for intellectual property pro- 
tection and disposition. However, the institution’s 
intellectual property policy is not formally evaluated 
during the peer review process or reviewed by the 
Ministry for adequacy. 


In our follow-up report to our 2003 value- 
for-money audit of the science and technology 
programs, the Ministry informed us that it had 
established a Commercial Advisory Council whose 
key tasks included reviewing intellectual property 
barriers to commercialization, with the expectation 
that a consistent policy for science and technol- 
ogy programs would result. However, during our 
current audit, we were informed that, owing toa 
realignment of government ministries, the council 
met only once, and no results ever materialized. 

We noted that several other jurisdictions, such 
as Quebec, Ireland, and Australia, have developed 
common guidelines for managing intellectual 
property for publicly funded research. The overall 
purpose of these guidelines is for the government to 
promote consistent good practices across research 
institutions for intellectual property management 
and commercialization of research results, while 
maximizing the benefits of publicly funded research 
to taxpayers and the economy. All of these jurisdic- 
tions promote the practice of placing intellectual 
property ownership with the research institutions, 
as opposed to the researchers. Generally, this is 
seen as good practice because institutions have the 
appropriate resources and experience to manage 
intellectual property and will do so for the benefit 
of the local economy, whereas individual research- 
ers who own intellectual property could move to 
another jurisdiction. In addition, having one party 
with title to the intellectual property is an import- 
ant incentive for industry and businesses that are 
interested in using research discoveries. 


RECOMMENDATION 2 


To better promote the commercialization of 
research done at Ontario’s publicly funded 
research institutions and ensure that the social 
and economic benefits of the research are 
retained in Ontario, the Ministry of Research 
and Innovation should continue to review best 
practices for intellectual property management 
in other jurisdictions and, on the basis of the 
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best practices identified, implement consistent 
guidelines for the management of intellectual 
property across Ontario’s publicly funded 
research institutions. 


The most effective approach to managing intel- 
lectual property (IP) remains an ongoing topic 
of debate within the research community across 
Ontario and Canada. 

The Ministry will continue to actively review 
best practices pertaining to IP management 
that are consistent with the Ontario Innovation 
Agenda. 

The Ministry will continue to work with uni- 
versities, research institutions, industry, and the 
financial sector to address issues of IP policy and 
management and encourage the development 
of IP models and approaches that will maximize 
the benefits of research programs to Ontario. 

The Ministry acknowledges the various 
approaches used by Ontario’s research institu- 
tions to manage IP and recognizes noteworthy 
examples where best practices for IP manage- 
ment have been implemented in institutions 
across Ontario. 


Measuring and Reporting on Program 
Effectiveness 


It is important for the Ministry to demonstrate that 
its programs are effective and how they provide 
value to Ontario. Well-defined program objectives 
are the basis for developing specific performance 
measures. Measuring and reporting on program 
performance is meant to guide decision-making 
and to demonstrate the Ministry’s accountability for 
achieving results. 

We reviewed the performance-measurement 
process for the Fund and found that there were only 
limited requirements for reporting results. As part 
of the government’s business-planning process, 
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the Ministry is required to report annually on the 
aggregate results for all its programs, including 
the Fund. Specifically, the Ministry reports on how 
its programs are performing against three stated 
targets: leveraged investment; the number of indi- 
viduals with enhanced skills involved in Ministry- 
funded projects; and active licences that resulted 
from Ministry-funded projects. 

In the February 2004 meeting of the Standing 
Committee on Public Accounts to review our 2003 
audit report on the Science and Technology Div- 
ision, the Ministry stated that “we are committed 
to measuring the contribution our programs make 
in promoting the growth of high-paying jobs.” The 
Ministry also said that the objective of the program 
was “to create knowledge, add to high-value jobs 
and create a climate that fosters commercialization 
of research.” 

The Ministry recently completed an inter- 
jurisdictional review of performance measures and 
is planning to develop a number of indicators for the 
public reporting of research and commercialization 
results. Although some performance measures have 
been developed, the Ministry does not measure or 
report publicly on the Fund’s achievement in sup- 
porting its goal of creating high-paying jobs. 

The Ministry produces a report on the results of 
the Research Infrastructure Program from data col- 
lected by the federal Canada Foundation for Innov- 
ation (CFI). Some key information is captured, such 
as the number of patents resulting from projects the 
CFI and Ontario have funded, but this information 
is not reported publicly. 

Although these measures give some indica- 
tion of the results being obtained for some of 
the research grants provided, the Ministry has 
not established any measurable targets or goals 
against which performance can be benchmarked. 
We were informed that, as part of a ministry-wide 
performance-measurement review, the Ministry 
will be redeveloping performance measures and 
setting performance targets for each of its pro- 
grams, including the Fund. 


RECOMMENDATION 3 





To improve its accountability to the public and 
its ability to measure the results being obtained 
for the grants provided by the Ontario Research 
Fund (Fund), the Ministry of Research and 
Innovation should: 

e develop program-specific measures, targets, 
and benchmarks to assess the Fund’s con- 
tributions to its overall goals of supporting 
job creation and the commercialization of 
research; and ; 

@ periodically report to the Legislature and 
the public on the achievement of these 
measures. 





The Ministry is reviewing its existing perform- 


ance measures with the goal of improving 
program-specific and impact-focused perform- 
ance measures. Our objective is to improve the 
Ministry’s ability to measure the impact of the 
Fund’s programs in terms of strategic objectives 
and government priorities. 3 

This objective is in keeping with the mandate 
of the Ontario Innovation Agenda, which calls 
for effective program measures to be in place for 
all key research and innovation programs. 

In the next phase of the development of 
the Fund, the Ministry is committed to imple- 
menting a system of program evaluation to 
measure the economic and societal impact of all 
Ontario Research Fund programs. 


PROJECT SELECTION 


The Ministry is ultimately responsible for ensuring 
that the selection of potential research projects 

best achieves the Fund’s goals and complies with 
specified eligibility criteria. To ensure that funds 
are provided to the most deserving proposals, a fair, 
effective, and transparent selection process must 


be used. We reviewed the selection process for the 
Research Excellence Program and the Research 
Infrastructure Program. 


Research Excellence Program 


The Research Excellence Program focuses on 
scientific excellence, strong commercialization, 
and strategic value to Ontario, and targets new, 
leading-edge research initiatives. The program sup- 
ports the operating costs of research, including an 
indirect cost component of up to 40% of a project’s 
direct costs. Total funding for individual projects is 
derived from equal contributions by the Ministry, 
the research institution, and the private sector. The 
Research Excellence Program generally requires 
one-third of the project costs to be funded through 
private-sector contributions. The intent of this 
requirement is to encourage partnerships between 
research institutions and the private sector, and to 
encourage commercialization. As of March 2009, 
the Minister had approved 71 Research Excellence 
Program projects, for a total ministry commitment 
of $306 million, with research institutions and 
private-sector partners committed to contribute 
additional funding of $309 million and $271 mil- 
lion, respectively. 

Peer review panels evaluate all Research Excel- 
lence Program proposals to help in the selection of 
projects that best meet the program-eligibility cri- 
teria. On the basis of advice given by the panels, the 
Ontario Research Fund Advisory Board, made up of 
senior level executives from the academic, govern- 
ment, research, and business communities, makes 
funding recommendations to the Minister, who 
makes the final decision on the proposals that will 
be funded. We reviewed the Research Excellence 
Program selection process and found that most 
projects met program-eligibility criteria and went 
through a review process that was fair and trans- 
parent. However, we noted a few major exceptions. 

Program guidelines state that the research 
funded should remain sustainable after provincial 
funding is no longer available. We found that the 
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Ministry did not always ensure compliance with 
this eligibility criterion. Specifically, the Ministry 
awarded $40.5 million (Research Excellence Pro- 
gram—$30.5 million; Research Infrastructure Pro- 
gram—$10 million) to three research facilities that 
provide computing services to researchers through- 
out Ontario. The research facilities funded include 
an operational and infrastructure component, and 
utilize large supercomputers that help provide sup- 
port to research in a broad range of disciplines from 
economics to biomedical engineering. Although 
ministry staff informed us that these facilities are 
necessary to support research in the province, they 
are not sustainable without continued govern- 
ment funding. Two of these facilities had received 
$23.7 million in provincial funding from previous 
research programs. 

The Ministry was not always consistent in apply- 
ing the program-eligibility criteria to ensure that 
only eligible projects were selected. Specifically, a 
laboratory was provided funding of $23.5 million 
(Research Excellence Program—$17.9 million; 
Research Infrastructure Program—$5.6 million) on 
the basis of its strategic value to Ontario. Although 
it was given an outstanding rating for commercial- 
ization by the peer review panel, the Advisory Board 
noted that the laboratory lacked commercialization 
potential and it was questionable whether it was a 
defined project under the guidelines. The project 
was then referred by the Advisory Board to the 
Minister for a final funding decision. We noted 
that, in situations such as these, there may be more 
appropriate funding sources. For example, the Min- 
istry awarded $15 million through another funding 
source to a similar proposal that had been rejected 
for funding through the program because there was 
no expectation of commercialization in the immedi- 
ate future. 


RECOMMENDATION 4 


To ensure that the Research Excellence Program 
follows a selection process that is not only fair 
and transparent but promotes the program’s 
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goals, the Ministry of Research and Innovation 
should ensure that all approved proposals meet 
program-eligibility requirements. 





To date, all projects that have received fund- 


ing through the Fund have gone through the 
adjudication and selection process. The Ministry 
will continue to ensure that all proposals are 
reviewed against program-eligibility require- 
ments as part of the approved adjudication and 
selection process by the Fund’s Advisory Board. 
The Board will continue to refer any exceptions 
to the Minister for final decision. 


Research Infrastructure Program 


The Research Infrastructure Program ensures that 
Ontario’s publicly funded research institutions 
continue to have competitive, state-of-the-art 
infrastructure to engage in world-leading research 
and technology development. This program has 
two components, the Large Infrastructure Program 
and the Small Infrastructure Program. The Large 
Infrastructure Program is directed toward large, 
strategic investments in research facilities that 
stimulate technology development, while the 
Small Infrastructure Program is directed toward 
investments that help attract, retain, and develop 
researchers by providing funds needed to keep their 
laboratories and equipment up to date. 

The Research Infrastructure Program is based 
on a co-funding model with the Canada Foundation 
for Innovation (CFI), a corporation created by the 
federal government to fund research. The program 
allows research institutions to leverage federal 
infrastructure awards toward projects that advance 
Ontario’s innovation goals and priorities. The prov- 
ince typically contributes up to a maximum of 40% 
of eligible costs, while the CFI contributes 40%, 
and the research institution and private funding 
partners fund the remaining 20% of a project’s cost. 


As of March 2009, the Minister had approved 995 
Research Infrastructure Program projects for a total 
ministry commitment of $317 million, with the 

CFI and other partners (research institutions and 
the private sector) contributing additional funding 
commitments of $457 million and $293 million, 
respectively. 

The peer review process that evaluates the sci- 
entific merit of research infrastructure proposals is 
managed by the CFI. To avoid duplication of efforts, 
the Ministry generally relies on the CFI’s decisions 
regarding which projects should be funded. For the 
Large Infrastructure Program, the Ministry informs 
the CFI of proposals that it believes best meet 
Ontario’s goals and should be funded. On the basis 
of the CFI’s funding decisions, the Advisory Board 
makes funding recommendations to the Minister, 
who makes the final decision on the proposals to be 
funded. 

We reviewed this process and found that the 
Ministry generally approved those projects that the 
CFI had decided to fund and did not always ensure 
that the proposals selected supported Ontario’s 
strategic goals, such as the commercialization of 
research. More specifically, for the Large Infra- 
structure Program, the Ministry matched the CFI 
funding decisions for 40% of funded projects worth 
$41.5 million that the Ministry did not recommend 
as being highly aligned with Ontario’s priorities, 
and the CFI rejected funding for proposals worth 
$18 million that the Ministry had identified as 
highly aligned with Ontario’s strategic priorities. We 
were advised that for the 2009 round of Large Infra- 
structure Program proposal selections, the Ministry 
intends to review all applications to assess the stra- 
tegic benefits to Ontario and does not plan to auto- 
matically match funding of CFI-selected projects. 
The Ministry will continue to match the CFI award 
decisions for the Small Infrastructure Program. 

In addition, to be considered for Research Infra- 
structure Program funding, the Ministry requires 
research institutions to apply to both the CFI and 
Ontario. However, we noted that in two other juris- 
dictions, British Columbia and Alberta, to ensure 


that projects with substantial provincial benefits 
are funded, research institutions are able to submit 
applications and receive funding even if they are 
not associated with a federal CFI application. 


RECOMMENDATION 5 


To ensure that projects funded by the Research 

Infrastructure Program are economically bene- 

ficial to Ontario, the Ministry of Research and 

Innovation should: 

e only fund projects that are highly aligned 
with Ontario’s priorities; and 

@ consider funding projects that have not 
applied to, or received funding from, the 
Canada Foundation for Innovation, if they 
offer significant benefits to Ontario. 


In the most recent round of the Ontario Re- 
search Fund—Large Infrastructure competition, 
the Ministry established Strategic Value Peer 
Review panels to make recommendations to the 
Fund’s Advisory Board on the extent to which 
proposals were aligned with Ontario’s stra- 
tegic priorities. As a result of this process, the 
Ministry will fund only those projects that are 
aligned with Ontario’s priorities. 


Colleges and Smaller Institutions 


There are 24 colleges in Ontario, with operations 
in over 100 Ontario communities. According to the 
Ontario Innovation Agenda, Ontario’s community 
colleges are highly responsive to the needs and 
interests of local communities and industries, mak- 
ing them an important link in responding quickly 
to changing skill requirements. Many colleges also 
work closely with small- to medium-sized enter- 
prises (SMEs) in particular, using their capacity 
for applied research and development to solve a 
company’s problems. This provides colleges with 
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an important role in contributing to Ontario’s eco- 
nomic growth, as over 99% of businesses in Ontario 
are SMEs. 

Traditionally, research funds have been more 
accessible to large universities than to smaller 
universities and colleges. Since the inception of 
the Fund, most of its funding has been allocated to 
universities, with three universities receiving over 
40% of total program funding. Although colleges 
are eligible to apply, no funding has been directly 
awarded to an Ontario college through the Fund. 
We found that colleges have been excluded from 
Small Infrastructure Program funding because the 
CFI specifically excludes colleges from receiving 
program funding (although at the federal level 
another agency has a program dedicated to funding 
colleges). 

We contacted half the colleges in Ontario to 
obtain feedback on how the program could be 
improved to be more accessible to colleges. All 
agreed that Ontario would benefit from a provincial 
research program dedicated to colleges and smaller 
institutions that addressed their unique needs and 
infrastructures. Also, the majority of respondents 
said their college has close ties and partnerships 
with SMEs. For example, one college indicated 
that over half the research it conducts is based on 
specific requests from SMEs, while another college 
informed us that it had over 40 ongoing projects 
with SMEs. 

Colleges do receive some funding from other 
ministry programs such as the Ontario Research 
Commercialization Program (ORCP) and the 
Ontario Centres of Excellence. For example, the 
government announced funding of $10 million over 
three years from the ORCP to the Colleges Ontario 
Network for Industry and Innovation to help SMEs 
with hands-on applied research, technology trans- 
fer, and commercialization. However, the colleges 
we spoke to indicated that these programs do not 
provide continuous long-term support to build the 
colleges’ research capacity. 
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RECOMMENDATION 6 


To ensure that the Ontario Research Fund 
selection process is accessible to all eligible 
applicants, and to help meet the program’s 
overall goal of commercialization of research, 
the Ministry of Research and Innovation should 
work with colleges, smaller institutions, and 
federal research agencies to ensure that the 
specific requirements and infrastructure needs 
of Ontario colleges and smaller institutions that 
focus on applied research are given appropriate 
consideration. 





The Ministry recognizes that colleges and 


smaller institutions have exhibited a lower suc- 
cess rate in Fund competitions than larger insti- 
tutions. The Ministry will continue to conduct 
outreach with colleges, smaller institutions, 
and federal research agencies to ensure that the 
needs of these stakeholders are understood. In 
addition, the Ministry will continue to look for 
opportunities to strengthen research capacity 
in colleges and smaller institutions through the 
Fund and other programs, such as the Colleges 
Ontario Network for Industry and Innovation, as 
noted in the Auditor General’s report. 


PROJECT MONITORING 


Monitoring is an ongoing process that may include 
activities such as site visits, phone and written 
inquiries, the review and analysis of reports, and 
follow-up of information received. The financial 
and operational performance of grant recipients 
should be reviewed and monitored throughout 

the grant cycle to ensure that program objectives 
are achieved efficiently and effectively and public 
funds used responsibly. The Management Board of 
Cabinet directive, Transfer Payment Accountability, 
also requires ministries to communicate with grant 


recipients on a regular basis, monitor the results 

for contracted projects, and take corrective action 
where a grant recipient has failed to meet contrac- 
tual obligations. Appropriate and timely monitoring 
is necessary to ensure that public funds are used 

for the purposes specified in the agreements and 
unused funds returned to the province. 

In our 2003 audit of the Science and Technol- 
ogy Division of the former Ministry of Enterprise, 
Opportunity and Innovation, we reported some sig- 
nificant concerns over the governance and account- 
ability mechanisms in place for the management 
of various Ontario research programs. Specifically, 
the Ministry had provided the Ontario Innovation 
Trust with $750 million to fund the capital costs of 
research in Ontario, but the Ministry was receiving 
virtually no information from the Trust and did not 
have the required monitoring processes in place 
to hold the Trust accountable for its expenditures 
of public funds. Also, the Innovation Institute of 
Ontario, a not-for-profit corporation responsible 
for the administration of the operational costs of 
research, did not retain research proposal assess- 
ments to support the decisions it made to fund 
specific projects. 

We believe that consolidation of operating and 
capital research funding into one comprehensive 
program that is managed and administered by 
the Ministry has improved the Ministry’s ability 
to effectively oversee its research grant program. 
Nevertheless, in our review of the Ministry’s 
monitoring processes for both the Research Excel- 
lence Program and the Research Infrastructure 
Program, we noted areas where improvements 
could be made. 


Research Excellence Program 


To assist in monitoring recipients of Research 
Excellence Program grants, the Ministry receives 
quarterly requests for payment, annual progress 
reports (APRs), independent financial audit 
reports, and reports on performance measures from 
grant recipients at various stages of their projects. 
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We reviewed the monitoring process and noted the broader impacts and scientific achievements. 


following: Overall, APRs were being received, although 


e The request for payment is an important many were submitted late. Ministry staff 


tool that the Ministry uses to monitor the 
financial progress of a project; consequently, 
grant agreements require recipients to report 
and request payment on a quarterly basis. 
Specifically, the Ministry uses the quarterly 
submissions to ensure that the private sector 
is contributing its share of project costs and to 
determine if project expenses are in line with 
the original budget. Monitoring private-sector 
contributions tells the Ministry if a project 
has industry support. According to the grant 
agreement, the Ministry’s commitment is con- 
tingent upon the private sector’s fulfilling its 
share. From our sample of projects, we found 
that 65% of recipients had not submitted 
requests on a timely basis (that is, quarterly); 
on average, requests were six months late. 
Although ministry staff generally followed 
up, this was normally done five months after 
the recipient should have filed its quarterly 
report. Also, in half the projects we reviewed, 
the private sector was under-contributing its 
share of project costs. The private sector had 
agreed to contribute $35 million to these pro- 
jects, but only $21 million had been provided, 
resulting in a shortfall of $14 million. Program 
staff informed us that some of the projects 
had start-up or operational delays, others had 
lost private-sector funding and were finding 
replacement funds, and many were delayed in 
reporting because collaborating institutions 
had not reported to the lead institution. More 
timely and thorough follow-up is needed to 
determine if the program should continue 

to fund projects that are not meeting their 
commitments. 

APRs are required from recipients to deter- 
mine whether a project is meeting its mile- 
stones and deliverables as agreed to in the 
grant agreement, and whether the project is 
on schedule. APRs also describe the project’s 


generally followed up with grant recipients, 
but the first reminder usually came six months 
after a report was due. We found the Ministry 
had not performed formal site-monitoring 
visits to assess the progress of the funded pro- 
jects. Because of the technical and scientific 
nature of the research projects, independent 
expert verification may be helpful, especially 
on the larger projects, to assess whether 
projects are progressing as intended. Many 

of the most prestigious academic and private 
research facilities in the United States have 
recently been accused of serious impropri- 
eties. Some of the more common impropri- 
eties related to research grants are falsifying 
progress reports, research data, results, and 
other documentation, and using grant money 
for other unrelated research or personal 
expenses. Routine site visits and independ- 
ent expert assistance could help prevent and 
identify such situations. As well, it would send 
a message to the research community about 
the Ministry’s concern that research funding 
be used only for the approved purpose. 

The Ministry receives independent financial 
audit reports at various times during a pro- 
ject, depending on the value of the grant. 
Generally, projects over $5 million require 

an audited annual report. For projects under 
$5 million the requirements vary, but at least 
one final report is required. However, the 
Ministry has not developed clear terms of ref- 
erence on the required contents of the audit 
reports and what the audits are expected to 
accomplish. We reviewed several audit reports 
submitted by recipients and found that most 
of the audit reports were submitted late and 
the content of the reports was inconsistent. 

A few of the reports briefly described the 
basis of accounting used, but most did not. 

In one-third of the reports reviewed, revenue 
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and expenses could not be verified by the 
accountant, but the Ministry did not follow 
up. Generally, the usefulness of these reports 
for monitoring where grant funds were spent 
was limited. 

The Research Excellence Program is 
intended to fund the operating costs of 
research; consequently, program guidelines 
recommend that the costs of facilities and 
equipment not exceed 10% of a project’s total 
direct costs. Also, to ensure that reasonable 
rates are charged for salaries and benefits, _ 
program guidelines limit the rates that should 
be charged according to the credentials of 
the researcher. We noted several instances 
where the Ministry approved projects that had 
estimated expenditures in these areas that 
exceeded guideline amounts, resulting in pos- 
sible total ineligible program funding of over 
$4 million. 


RECOMMENDATION 7 





To ensure that Research Excellence Program 
grants are used for the purposes intended and 
that project performance is effectively mon- 
itored, the Ministry of Research and Innovation 
should: 

e implement a process to identify and follow 
up on projects that are not reporting quar- 
terly as required; 

e@ perform routine, formal monitoring visits to 
verify the information submitted by grant 
recipients, to ensure that program funds are 
being used for the approved research and 
that research milestones have been met; and 

e develop clear guidelines for what independ- 
ent audits are expected to accomplish and 
report, ensure that audit reports are received 
when due, and follow up on issues they iden- 
tify on a timely basis. 


The Ontario Research Fund is a maturing pro- 
gram. The Ministry is committed to continually 
improving and refining the program’s delivery 
model. The Ministry agrees that projects should 
be effectively monitored. To assist in this 
regard, the Ministry has recently completed the 
development of a contract—-monitoring module 
for the Research Excellence program through 
the Research Awards Database, which captures 
key information for monitoring projects. 

In addition, as part of the Ministry’s 2008/09 
Audit Plan, Internal Audit conducted an audit 
of two large program recipients to assess the 
effectiveness of the Ministry in monitoring the 
recipients; to assess whether the recipients have 
achieved contract deliverables; to verify that 
the expenses incurred are eligible under the 
contract; and to confirm that there is adequate 
supporting documentation in place. The results 
of this review will be used to help develop 
enhanced program-monitoring processes. 


Research Infrastructure Program 


The Research Infrastructure Program is a co-funded 
program delivered in partnership with the CFI. The 
program has approximately four staff to oversee 
and monitor 995 projects worth $317 million. To 
avoid duplication of effort, the Ministry generally 
relies on the CFI to monitor research infrastructure 
grants. 

Before making payments on a research infra- 
structure project, the Ministry reviews the payment 
request to ensure that the CFI has already made its 
related payment. The Ministry receives from the 
institutions a copy of the annual project reports and 
financial reports that the CFI requires to monitor 
project performance. 

We were advised that the CFI conducts periodic 
visits to assess the adequacy and effectiveness of 
policies, processes, and financial controls, and 


help to ensure that funds are being spent for their 
intended purposes. We were informed that the CF] 
also conducts contribution audits at the project 
level to ensure that the institutions have used the 
funds in accordance with the terms and conditions 
in the grant agreements and CFI guidelines and 
that private-sector contributions are valued cor- 
rectly and milestones are being met. 

The Ministry could request the results of the 
CFI’s monitoring visits and audits for Research 
Infrastructure-funded projects. However, we found 
that it had not requested or received this informa- 
tion for Ontario’s projects. Because the Ministry 
has not performed any of its own site visits, it is 
important to obtain such information to adequately 
assess whether further due diligence should be 
performed by ministry staff, or whether issues have 
been identified that affect the province. Also, we 
noted that the Ministry had no formal agreement 
with the CFI to clarify the roles and expectations of 
each of the funding parties and ensure the effective 
co-ordination of the oversight processes. 


RECOMMENDATION 8 


To more effectively monitor Research Infra- 
structure Program grants and ensure adequate — 
co-ordination of oversight processes with the 
Canada Foundation for Innovation (CFI), the 
Ministry of Research and Innovation should: 

@ periodically obtain and review the CFI mon- 
itoring reports and audits for selected larger 
Ontario-funded projects to ensure that prov- 
incial funds are being used for their intended 
purpose and funded institutions comply with 
program policies and guidelines; 

@ assess the need for ministry staff to conduct 
site visits, especially on the larger projects; 
and 

e establish a formal agreement with the CFI 
that clearly defines the roles and expecta- 
tions of each party in the oversight processes 
for co-funded projects. 
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The Ministry agrees that it would be helpful to 
work with the Canada Foundation for Innovation 
(CFI) to define a process for sharing information 
to facilitate oversight of co-funded projects. In 
particular, the Ministry will work with the CFI 

on selected larger Ontario-funded projects to 
ensure that provincial funds are being used for 
their intended purpose and funded institutions 
comply with program policies and guidelines. 


PROGRAM ADMINISTRATION 


The Ministry has approximately 15 full-time 
employees and five support staff dedicated to the 
administration and delivery of the Fund program. 
Since the inception of the program in 2004, the 
Ministry has paid out over $300 million in transfer 
payment grants and committed a total of $623 mil- 
lion to 1,066 projects. Financial and administrative 
controls are necessary to ensure that all of the 
program’s significant policies have been complied 
with and public funds are used with due regard 

for economy. We reviewed the Ministry’s financial 
and administrative controls over the program to 
determine if they were operating effectively and 
efficiently. We noted some areas where improve- 
ments could be made. 


Information Systems 


The Ministry uses a combination of spreadsheets, 
a database, and word-processing documents to 
manage Fund grants. The Research Award Data- 
base that the Ministry currently uses to manage 
grants has worked well as a repository for basic 
project information such as project description, 
funding amounts, and panel review assess- 
ments. However, the system is outdated and is 
not able to produce essential program-level and 
project-specific information needed to manage 
the growing volume and complexity of the 1,066 
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grants awarded to date. We found that, although 
the Research Infrastructure Program relies on the 
system to monitor grant payments for individual 
projects and the program as a whole, the Research 
Excellence Program does not. Also, the database 
does not capture key information for monitor- 

ing the performance of specific projects against 
contract terms, such as outstanding request for 
payments, due dates, and private-sector contribu- 
tions. A contract-monitoring module is being built 
into the system to track project-specific informa- 
tion, but it is limited to the Research Excellence 
Program and, at the completion of our field work, 
was not yet operational. 

The Ministry is developing a new e-grants sys- 
tem with a total estimated cost of $2.9 million to 
help manage grants for all its research programs. 
The project is still in the planning phase; the new 
system is expected to be rolled out in April 2010 for 
another ministry program, but no date has been set 
for the Fund. In addition, the contract-monitoring 
component for the new system has not been 
designed, and as of March 31, 2009, the Ministry 
had not estimated the cost of this component. 





RECOMMENDATION 9 


To ensure that the Ministry of Research and 
Innovation has the information needed to 
effectively oversee its Ontario Research Fund 
program, its information system should provide 
ministry staff with timely program-level and 
project-specific information. 





The Ministry agrees that its information sys- 


tem should provide ministry staff with timely 
program-level and project-specific informa- 
tion. To accomplish this goal, the Ministry is 
developing an e-grants system. The e-grants 
system is a user-friendly and time-saving 
service-delivery and management tool that will 
automate the grant management processes and 


align the Ministry with the Ontario Innovation 
Agenda. In addition, the e-grants system will 
offer improved access for applicants and offer 
ministry staff improved access to information 
to manage programs, including the Fund, more 
strategically. 


Private-sector Partner Contributions 


Meaningful participation in research programs 

by the private sector is often an indication that 

the research being carried out has commercial 
relevance and value to industry. The private sec- 
tor’s interest in the research may be in the form of 
cash or in-kind contributions. In-kind contributions 
include such items as equipment, materials, use of 
facilities, and research personnel. Cash contribu- 
tions are more readily verifiable by the Ministry 
and could indicate a higher level of commitment for 
the commercial potential of research projects. As 

of March 31, 2009, approximately 60% ($164 mil- 
lion) of the Research Excellence Program and 99% 
($156 million) of the Research Infrastructure Pro- 
gram private-sector contributions had been made 
in kind, as opposed to cash contributions. 

Fair valuations of in-kind private-sector contri- 
butions are necessary to ensure that the Ministry 
is not contributing disproportionately to projects 
by matching contribution values that have been 
inflated. In its February 2004 meeting with the 
Standing Committee on Public Accounts, the 
Ministry stated that it would “develop policies to 
confirm contributions and ensure that there is 
independent valuation of in-kind contributions of a 
designated material value.” 

We reviewed reported private-sector in-kind 
contributions of over $65 million to determine 
whether the Ministry had established policies to 
ensure that contributions were properly valued. 
We found little evidence on file of independent 
evaluations to confirm that the actual value of 
in-kind contributions was equal to the value being 


claimed by the contributor. For the Research Infra- 
structure Program, the Ministry relies on the CFI 

to ensure that reliable valuations are performed. 
For the Research Excellence Program, it follows the 
CFT’s policy, which normally requires some form 

of independent third-party evaluation, depending 
on the type and value of the in-kind contribu- 

tions. For example, the policy requires competitive 
quotes through a formal bid process or third-party 
appraisal for in-kind contributions of equipment 
greater than $500,000. However, we noted several 
specific examples where there was no independent 
evidence on file to support the reported values. In 
one case, the equipment was valued at $4.8 million. 
In another example, the private sector contributed 
$18 million for facilities and equipment—but again, 
we found no independent evaluations on file to sup- 
port the stated value of this in-kind contribution. 
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RECOMMENDATION 10 


To provide assurance that in-kind private-sector 
contributions are fairly valued, the Ministry of 
Research and Innovation should: 

@ ensure that grant recipients comply with the 
policies adopted for the program to assess 
the value of in-kind contributions; and 

@ periodically verify that independent valua- 
tions of substantial in-kind contributions 
have been performed to support values 
reported by grant recipients. 


The Ministry, through the Fund, will continue 
to evaluate in-kind contributions from private- 
sector contributors in accordance with best 
practices established by peer-funding agen- 
cies. In addition, the Ministry will ensure that 
periodic verification of in-kind contributions is 
performed. 
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The Ministry of Community and Social Services 







Cha pter i} Ministry of Community and Social Services 
Section 


eee Ontario Works Program 


is intended to help eligible applicants with basic liv- 
Background ing expenses such as food, clothing, personal needs, 
and shelter. Employment assistance for eligible 


applicants includes a variety of activities intended 


ae ; E : to increase their employability and help them 
(Ministry) provides social assistance under two } : 
; ee obtain employment and become self-reliant. 
programs to approximately 450,000 individuals ae ; 
airs Basic income assistance under Ontario Works is 
as well as their qualifying family members for a 
ay generally less than comparable payments under the 

total of over 700,000 people. Under the provisions eae ee 
Fae ae ae Ontario Disability Support Program. A comparison 
of the Ontario Disability Support Program Act, 
of typical monthly benefits, all of which are tax- 
income and employment supports are provided to oe 
} bcs ; free, between the time of our last audit in 2002 and 
approximately 250,000 individuals with eligible 


disabilities, as defined by the act. Under the Ontario 
Works Act, the subject of this audit, the Ontario 
Works program provides income and employment 


the 2008/09 fiscal year appears in Figure 1. 
In addition to income assistance, Ontario Works 
recipients also may be eligible for benefits for a 


: seein ; number of other items to assist in specific circum- 
assistance to approximately 200,000 individuals in ; : 
; stances on the basis of established need. These 
temporary financial need, who are unemployed or aad 
. ; : include: 
underemployed. Ontario Works income assistance 





Figure 1: Maximum Monthly Ontario Works Benefits (Tax-free) 
Source of data: Ministry of Community and Social Services 





Couple With One 
Single Person Spouse Disabled and 
Single Person With One Child Two Children 
basic needs allowance ($) 7 195 a 216 é 446 | 360 576 429 
maximum shelter allowance ($) 325 356 511 560 602 660 





comparable ODSP benefit ($) Eo 930 1,020 1,424 





1,423 1,816 1,680 


* reduction due to the introduction of the Ontario Child Benefit 


e health-related necessities, including medical 

supplies, and basic dental and vision care; 

@ community start-up benefits to assist in the 

establishment of a permanent residence; and 

e@ employment start-up and participation- 

related expenses, including transportation, 
training fees, and clothing. 

To be eligible for assistance, applicants must 
demonstrate financial need by providing evidence 
that their non-exempt liquid assets and income levels 
fall below specified amounts. In addition, applicants 
also are required to sign an agreement to participate 
in one or more activities designed to gain skills and 
progress toward sustainable employment, unless 
granted a deferral for medical or other reasons. 

The Ontario Works program is delivered on 
behalf of the Ministry by 47 Consolidated Munici- 
pal Service Managers and District Social Services 
Administration Boards as well as 100 First Nations, 
referred to as service managers. A service manager 
is typically either a large municipality or a group- 
ing of smaller ones, and each one is accountable to 
one of the Ministry’s nine regional offices. Service 
managers have been designated the regulatory 
authority to make eligibility determinations. 

The Ministry and the service managers share the 
total financial and employment assistance costs of 
the Ontario Works program, as shown in Figure 2. 
The Ministry, which pays 80% of these costs, has 
committed to start gradually increasing its share 
in 2010 until it pays 100% in 2018. Administrative 
costs will continue to be shared on a 50/50 basis up 
to the approved budget. 

In the 2008/09 fiscal year, the Ministry’s share 
of income assistance provided to individuals 





Figure 2: Cost-sharing of Ontario Works Expenditures 
(2008/09) (%) 


Source of data: Ministry of Community and Social Services 


income assistance 


employment assistance 


administration 
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through Ontario Works was more than $1.5 bil- 
lion. The Ministry spent a further $194 million 
for program administration and $171 million on 
employment assistance programs. The Ministry’s 
total Ontario Works expenditure for 2008/09 was 
therefore about $1.9 billion. 

Since the time of our last audit in 2002, the pro- 
gram’s caseload has increased by 3% and the Min- 
istry’s share of program expenditures has increased 
on average by approximately 2% per year, as shown 
in Figure 3. The administration of Ontario Works is 
supported by the Ministry’s computerized informa- 
tion system, commonly referred to as the Service 
Delivery Model Technology (SDMT) system. The 
system was implemented province-wide in 2002. 


Audit Objective and Scope 





Our audit objective was to assess whether the 
Ministry’s policies and procedures for the Ontario 
Works program and its oversight of the Consoli- 
dated Municipal Service Managers were adequate 
to ensure that: 

@ only eligible individuals received the correct 
amount of financial assistance as well as 
appropriate employment assistance to help 
them find paid employment and become self- 
reliant; and 

@ the Ontario Works program was delivered 
with due regard for economy and efficiency. 

The scope of our audit included a review and 

analysis of relevant ministry files, policies, and 
procedures, as well as interviews with appropriate 
staff at the Ministry’s head office, at three regional 
offices (Toronto, Hamilton-Niagara, and Northern), 
and at three municipal service managers that we 
visited. Collectively, the three service manager 
offices we visited represented approximately 

40% of the Ministry’s total program cost. We also 
requested summary statistics and other information 
about the program from the 44 service managers 
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Figure 3: Ontario Works Caseload and Ministry Share of Expenditure 


Source of data: Ministry of Community and Social Services 

















# of Total 


Total Ministry 
Expenditure 





% Change Dependents Beneficiaries ($ million) % Change 
2001/02 196,596 222,897 419,493 1,669.2 
2002/03 195,137 (0.7) § 208,930 404,066 1,726.0 3.4 
2003/04 192,096 (1.6) 197,657 389,751 1,639.5 (5.0) 
2004/05 191,723 (0.2) 188,946 380,669 1,677.0 20 
2005/06 198,378 Se) 188,424 385,806 Pi53.2 45 
2006/07 199,242 0.4 183,826 383,068 1,794.5 2.4 
2007/08 194,920 (2.2) 176,955 SILO03 1,807.4 0.7 
2008/09 202,181 3.7 178,261 380,446 1,899.2 oes | 


that we did not visit, and over 80% responded to 
our request. 

We retained the services of an IT security spe- 
cialist to help us assess the security of the SDMT 
system and follow up on system-security issues 
identified in previous audits. We also met with 
senior representatives of a client stakeholder group 
that advocates on behalf of social assistance recipi- 
ents to obtain their perspective on the issues facing 
Ontario Works. 

We set objectives for what we wanted to achieve 
and developed audit criteria that covered the key 
systems, policies, and procedures that should be in 
place and operating effectively. We discussed these 
criteria with senior management of the Ministry, 
who agreed to them. We then designed and con- 


ducted tests and procedures to address our audit 


objectives and criteria. 

Although our audit work, particularly with 
respect to income and employment assistance, 
often covered a number of years, our findings 
emphasized the assessment of, and compliance 
with, the policies and procedures in place for the 
Ontario Works program during the 2007/08 and 
2008/09 fiscal years. 

We also reviewed a number of recent audit 
reports issued by the Ministry’s Internal Audit 
Services, many of which related to specific aspects 
of the SDMT system. These reports contained a 


number of findings that we considered in deter- 
mining the scope of our own review of the SDMT 
system. However, none of the reports concentrated 
on basic needs and shelter allowance, employment 
assistance, and program administration costs, 
which together constituted the main focus of our 
audit. We were, therefore, unable to rely on these 
reports to reduce the scope of the primary focus of 
our work. 


Although the Ministry has implemented a number 
of the changes we recommended in our 2002 audit, 
there has been limited improvement in the admin- 
istration of the Ontario Works program since that 
time. It remains our view that the Ministry still has 
inadequate assurance that only eligible individuals 
receive financial assistance and in the correct 
amount. Although the Ministry considers Ontario 
Works financial assistance to be a temporary meas- 
ure, about one-third of the recipients in the three 
municipal service managers’ offices we visited were 
receiving payments for longer than two years and 
some 13% for more than five years. 

An appropriate level of oversight is necessary if 
the Ministry is to have confidence that only eligible 


individuals receive financial assistance and in the 
correct amount. We found that further improve- 
ments in overseeing service delivery were needed. 
As well, given the size and scope of Ontario Works, 
the supporting information technology system must 
be reliable and provide the information needed to 
enable the program to be effectively managed. We 
again had concerns in this area. 

With respect to the Ministry’s oversight of 
Ontario Works program delivery by the service 
managers, our specific concerns included the 
following: 

e During the Ontario Works application process, 
municipal service managers rely on individ- 
uals to provide almost all the information 
used to determine their initial eligibility for 
income assistance. The risk of new applicants 
under-reporting their income and assets is 
compounded by the fact that the service man- 
agers seldom undertook the required third- 
party verifications—because they felt they 
were not necessarily required at the initial 
eligibility stage. 

e@ Many assistance recipients did not submit the 
required income reports every month—often 
failing to do so for extended periods of time— 
and we seldom found any documentation on 
file to indicate that this reporting stipulation 
had been waived. 

e Benefits for such things as community and 
employment start-up activities were often 
paid without any evidence that the activity 
had occurred and/or documentation to sup- 
port the amount reimbursed. Such payments 
also often exceeded established maximums. 

e@ Many applications for special dietary allow- 
ances were associated with questionable 
circumstances. For example, we found 
several instances where each member of a 
large family was diagnosed by a health-care 
practitioner with identical multiple medical 
conditions. As a result, in one example, a 
family of 10 people each received the max- 
imum special dietary allowance of $250 per 
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month, or about $30,000 a year for the entire 
family (combined with other allowances, such 
a family would receive approximately $50,000 
in a year, tax-free). The total amount spent on 
dietary allowances has increased from $5 mil- 
lion in the 2002/03 fiscal year, the time of our 
last audit, to more than $67 million during the 
2008/09 fiscal year. 

Unrecovered overpayments to approximately 
350,000 current and former Ontario Works 
recipients increased from $414 million in 
February 2002 to $600 million as of March 31, 
2009—a 45% increase. Efforts by service 
Managers to recover these overpayments had 
been minimal, possibly owing in part to the 
lack of financial incentive for them to do so. 
Many tips from the fraud hotline were either 
inadequately investigated or ignored. 

We found little evidence in recipient files to 
indicate that the service manager casework- 
ers were involved in determining the most 
appropriate employment assistance activity 
and there is no standard requirement to docu- 
ment this process. Rather, recipients usually 
selected the activity that they felt was best 
suited to get them back into the workplace. 
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Our province-wide analysis showed that two- 
thirds of recipients listed “independent job 
search” as the most beneficial activity. 

We found that the Ministry’s required reviews 
of a sample of service-manager files were 
being done on time and that the work was 
being reasonably well done. However, even 
though the Ministry was noting many of the 
same systemic file deficiencies that we identi- 
fied during our audit, there seemed to be little 
progress in addressing those deficiencies from 
one year to the next. 

The Ministry’s examinations of a sample of 
service managers’ reimbursement claims did 
not occur on an annual basis as required, nor 
did the Ministry ensure that submitted claims 
were complete, accurate, and based on actual 
payments made to recipients. These reviews 
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are critical given the fact that the Ministry’s 
subsidies totalled $1.5 billion in the 2008/09 
fiscal year. 

e The Ministry continued to reimburse service 
managers’ administrative costs on an histor- 
ical basis rather than on a formula based on 
costs per case. At the same time, the Ministry 
lacked the detailed information necessary 
to assess the reasonableness of the service 
managers’ expenditures for administration. In 
addition, some service managers felt that they 
were absorbing much more than their 50% 
share of approved administrative costs. 

e The Ministry had insufficient information to 
assess whether employment assistance funds 
were being used as intended and whether 
these expenditures were actually helping 
people obtain employment. 

e@ The Ministry had very little information avail- 
able to assess the efficiency and effectiveness 
of program delivery. However, the Ministry 
introduced outcome measures as a pilot 
project in 2006, to be reported on by service 
managers over two-year cycles. The first of 
those two-year cycles began in 2008 and 
required service managers to track perform- 
ance and assess employment strategies based 
on outcome targets. 

Despite improvements to the Ministry’s Service 
Delivery Model Technology information system 
since its rollout in 2002—many of which were 
intended to enhance reliability as well as the 
completeness and accuracy of its information—the 
system continues to have reliability concerns and 
known deficiencies. They included: 

e@ SDMT system users did not receive in an easily 
understandable format the information they 
needed to effectively manage and oversee 
the program. In addition, the system lacks a 
report-writing function that allows users to 
easily extract the information they need on an 
ad hoc basis. 

@ Service managers told us that they compen- 
sated for the SDMT system’s limitations by 


developing approximately 150 different work- 
around systems and processes. The service 
managers advised us that the development 
of many of these standalone workarounds 
incurred considerable costs and time. For 
instance, most service managers maintain 
standalone systems to manage the Ontario 
Works employment assistance function, a 
critical component of the program that the 
SDMT does not cover as comprehensively as 
required. 

e Although there is a reasonable level of 
security control to protect the system from 
external attacks, it is not adequate to prevent 
an internal system user with IT knowledge 
from escalating restricted access to full access, 
which increases the risk of fraudulent pay- 
ments being made. 

We understand that the government has desig- 
nated the SDMT system as a priority as part of its 
project to remediate high-risk applications. The 
Ministry is reviewing its options for potential sys- 
tem refinements or other opportunities to improve 
the system’s technology. It is to prepare a business 
case in this regard for late autumn 2009. 





The Ministry of Community and Social Services 


welcomes the findings and recommendations of 
the Auditor General with respect to the delivery 
and oversight of the Ontario Works program. 
This is a vital service for some of the most vul- 
nerable citizens of Ontario, particularly in the 
current economic times. The Ministry has taken 
steps over the past several years to improve 
program oversight and management, and will 
continue to focus efforts on improving program 
administration. 





Detailed Audit Observations 


MINISTRY OVERSIGHT AND CONTROL OF 
PROGRAM DELIVERY 


Program Delivery Overview 


In most cases, an individual starts the application 
process for Ontario Works benefits by visiting or 
telephoning the local municipal service manager. 
During this initial contact, service manager staff 
provide the individual with information regarding 
the application process, including eligibility criteria 
and the information and documentation required 
to complete the application. Service manager staff 
also obtain basic information about the individual 
such as name, address, age, number of dependants, 
and so on, which is entered into the SDMT system. 

A decision with respect to eligibility for assist- 
ance is made after the application process is 
completed at an in-depth, intake appointment. The 
Ministry’s goal is to have service managers conduct 
the appointment and make a decision within four 
days of the initial contact. 

For the application to be complete, applicants 
must provide all of the previously requested infor- 
mation and supporting documentation. To be finan- 
cially eligible, a person’s total non-exempt assets 
must be at or below: 

e@ $572 for a single person; and 

e $989 if there is a spouse in the benefit unit 

(family). 
(These amounts are generally increased by 
$500 for each eligible dependant.) 

Certain items, such as a principal residence, 

a primary vehicle valued at less than $10,000, a 
locked-in RRSP, and pre-paid funeral expenses, are 
excluded when determining whether the person’s 
assets are below the prescribed limit. To be eligible 
for even a partial Ontario Works benefit, 100% of 
the applicant’s total family non-exempt income 
must be less than the amount of the potential 
Ontario Works entitlement. After three months of 
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assistance, 50% of earned income and amounts 
paid under a training program are exempt as 
income when determining eligibility. 

Applicants, their spouses, and any other depend- 
ent adults in the benefit unit must sign a participa- 
tion agreement. This agreement requires that 
the individual takes part in selected employment 
assistance activities and makes reasonable efforts 
toward seeking and obtaining paid employment. 

If the applicant has provided all of the necessary 
information and documentation, a final decision is 
made and communicated in writing. 

A request for emergency assistance can be made 
at any time during the application process, and up 
to 16 days of emergency income assistance for basic 
needs and shelter may be provided before a full 
application is required to be completed. 

In cases where the application is denied, appli- 
cants can request an internal review within 30 days. 
The review, conducted by another caseworker or a 
supervisor, must be completed within 10 days of the 
request being received. If a review is not completed 
within 10 days, or the applicant is not satisfied with 
the internal review decision, he or she can appeal to 
the Social Benefits Tribunal, an independent body 
that operates at arm’s length from the Ministry and 
the service managers. 


Initial Financial Eligibility Assessment 


As noted previously, applicants must provide muni- 
cipal service managers with the necessary informa- 
tion to establish their eligibility for assistance and 
to determine the correct amount to be paid. They 
are required to provide a number of documents 
that, depending on the document, must be either 
visually verified and have its relevant details noted 
in the computer system or copied and placed on 
file. These documents include a Social Insurance 
Number card, Ontario Health Insurance card, birth 
certificate, and any other document considered 
necessary to verify a person’s identity and legal 
status in Canada. 
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To determine the correct amount of assistance 
to be paid, applicants also must provide such things 
as monthly bank statements, pay stubs, records of 
employment, vehicle ownership registration forms, 
and evidence of shelter costs incurred. 

Our review of a sample of files for individuals 
receiving Ontario Works benefits found that, in 
many cases, critical documents necessary to conclu- 
sively establish an applicant’s identity and/or legal 
status were either not visually verified or a copy had 
not been placed on file as required. 

For example, at one municipal service manager 
we visited, some 8% of all recipient files lacked a 
Social Insurance Number, an omission that makes 
it difficult, for example, to detect duplicate pay- 
ments. Similarly, in a number of instances, there 
was no evidence on file that the recipient’s proof of 
identity, date of birth, or legal status in Canada had 
been verified. 

There is also an obvious risk that applicants 
could understate their income or assets when seek- 
ing assistance. As well, there is no assurance that an 
applicant has provided a bank statement for each 
account in his or her possession or all the pay stubs 
relevant to determining financial eligibility. 

To help overcome this risk and help verify that 
the income and assets declared by applicants are 
complete and accurate, the Ministry has entered 
into a number of third-party, information-sharing 
agreements. These include arrangements with 
Human Resources and Skills Development Canada 
for employment insurance information, with the 
Canada Revenue Agency for tax return informa- 
tion, with Equifax for credit information, with 
the Ontario Ministry of Transportation for vehicle 
ownership information, and with the Family 
Responsibility Office to ascertain any support pay- 
ments received. 

The Ministry advised us that service managers 
must verify a recipient’s declared income and 
assets with these organizations at the time of initial 
eligibility determination and during all subsequent 
financial eligibility reassessments. However, staff 
at the three service managers we visited did not 


interpret the Ministry’s directives as requiring third- 
party verifications at the initial financial eligibility 
stage and, in practice, seldom undertook third- 
party verifications at that time. 

Although service manager staff acknowledged 
that third-party verifications were required at the 
time of a subsequent financial eligibility reassess- 
ment, we found that sometimes they were also not 
completed at that time. 

In our 2002 Annual Report, we highlighted our 
concerns over ineligible applicants possibly receiv- 
ing financial assistance. At the time, we said the 
Ministry “should reinforce with service managers 
its requirements for obtaining, documenting, and 
correctly assessing the required recipient informa- 
tion.” The Ministry advised us then, as well as in 
our follow-up in the 2004 Annual Report, that it 
would address this concern, yet many of the same 
issues are still not being adequately addressed. 


RECOMMENDATION 1 





To ensure that an individual’s initial finan- 
cial eligibility for Ontario Works benefits is 
adequately determined and that the correct 
amount of assistance is paid, the Ministry of 
Community and Social Services should make 
certain that Consolidated Municipal Service 
Managers: 

@ visually verify documents or obtain copies of 
all documents required to establish an indi- 
vidual’s identity and legal status in Canada, 
especially Social Insurance Number cards; and 

® comply in all cases with the requirement 
to verify an applicant’s declared income 
and assets with the third parties who have 
entered into information sharing agreements 
with the Ministry. 


The Ministry recognizes the need to ensure that 
only eligible persons are provided assistance 
through Ontario Works.and that the assistance 
provided is in the correct amount. 


The Ministry has program verification stan- 
dards in place to ensure initial and ongoing eli- 
gibility, and will take steps to ensure that service 
managers understand and comply with the pro- 
gram verification standards and requirements. 


Financial Eligibility Reassessments 


Ministry policy requires that municipal service man- 
agers reassess the continued financial eligibility of 
all their Ontario Works recipients at least once every 
12 months from the time of the last assessment. In 
doing so, service managers are expected to follow 
a Ministry-developed Consolidated Verification 
Process (CVP) checklist that requires completion of 
most of the same procedures, including third-party 
verifications, that were to have been undertaken 
during the initial financial eligibility assessment. In 
addition, the SDMT information system flags each 
recipient’s priority for an eligibility reassessment as 
high, medium, or low, on the basis of programmed 
risk factors. Service managers are expected to pri- 
oritize and complete the CVPs accordingly. 
However, we found that financial reassessments 
were not conducted at least once every 12 months 
in approximately half the files we sampled. In fact, 
in some instances, a financial eligibility reassess- 
ment had not been completed for up to five years. 
In one such case, undeclared income for a recipient 
that could have been detected through a third-party 
verification resulted in a $38,000 overpayment 
over a four-and-a-half-year period. No CVP review 
was conducted during that time, so the recipient’s 
income was not checked with the Canada Revenue 
Agency, as required by ministry policy. 
Furthermore, service manager staff did not 
follow the Ministry-prescribed CVP checklist about 
one-quarter of the time. Regardless of whether 
the checklist was used, the necessary documenta- 
tion was not on file in many cases to demonstrate 
that staff had adhered to the CVP requirements. 
Compounding our concerns over the lack of proper 
reassessment, we found that third-party verifica- 
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tions were not being conducted about one-third of 
the time at one service manager we visited. 

We also note that none of the three service man- 
agers we visited used the SDMT system risk flags or 
were not using them as intended to identify high- 
risk recipients so that CVPs could be conducted on 
them first. However, one of the three service man- 
agers had developed its own risk-ranking system 
that it thought was more effective. 


RECOMMENDATION 2 


To ensure that recipients continue to be finan- 
cially eligible for Ontario Works benefits and to 
avoid overpayments, the Ministry of Community 
and Social Services should make certain that 
Consolidated Municipal Service Managers: 

e@ complete financial reassessments on each 
recipient at least once every 12 months as 
required; 

@ use the Ministry-prescribed checklist when 
conducting a financial reassessment and 
obtain sufficient documentation, including 
third-party verifications, to support the out- 
come of the review; and 

@ help ensure that the risk flags in the Service 
Delivery Model Technology system are 
effective and are used to prioritize high-risk 
cases for review. 


The Ministry agrees that appropriate action 
should be taken to verify ongoing eligibility. 

The Ministry has program verification stan- 
dards in place to ensure initial and ongoing eli- 
gibility, and will take steps to ensure that service 

3 managers understand and comply with program 
verification standards and requirements. 

In addition, the Ministry will be imple- 
menting a risk-based approach to Ontario 
Works financial eligibility reassessments. This 
risk model will help to ensure that only eligible 
recipients remain on the program and that they 
receive the correct payments. 
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Other Income Reporting 


As was the case at the time of our last audit, recipi- 
ents are required to report income on a monthly 
basis—including changes in income—to help 
service managers determine ongoing eligibility for 
assistance and the correct amount of assistance to 
be paid. For recipients that have been on assistance 
for three months, 50% of their earned income or 
amount paid under a training program is deducted 
from the amount of their assistance. Non-exempt 
income from all other sources is deducted at a rate 
of 100%. However, service managers now have the 
discretion to waive the monthly income-reporting 
requirement for recipients who, according to one of 
the program’s directives, “have no income to report 
or have a static or fixed income.” 

Our review of a sample of case files found that 
monthly income reports were frequently not submit- 
ted for many months, or not at all. In these cases, 
there was no evidence that the service manager had 
waived the monthly income-reporting requirement. 
One service manager indicated that its practice was 
to generally waive the income-reporting require- 
ment in all cases—requiring the report on an excep- 
tion basis only—and that there was therefore no 
need to document the waiver in each recipient file. 
However, in the absence of a waiver or any follow- 
up, it was unclear whether the recipient had income 
that should have been considered in determining 
the following month’s entitlement. In that regard, 
we noted that the external auditor of one service 
manager found that over 60% of overpayments in 
a sample reviewed had resulted because recipients 
had failed to report their income. 





_ RECOMMENDATION 3 





To ensure that financial assistance provided by 
Ontario Works is in the correct amount and to 
minimize overpayments, the Ministry of Com- 
munity and Social Services should make certain 
that Consolidated Municipal Service Managers 
receive a monthly income report from each 


recipient, unless they waived the requirement 
for sound reasons that are documented on file. 
If it is the Ministry’s intention that Consolidated 
Municipal Service Managers require the report 
on an exception basis only, that should be more 
clearly communicated. 


The Ministry recognizes the need to ensure that 
only eligible persons are provided assistance 
through Ontario Works and that the assistance 
provided is in the correct amount. The Ministry 
will reassess the current requirements for 
income reporting. Following this reassessment, 
we will clearly communicate requirements and 
reinforce service managers’ compliance. 


Other Financial Assistance and Benefits 


Ontario Works recipients may be eligible to receive 
supplemental income assistance or benefits in addi- 
tion to assistance for basic needs and shelter. The 
most common supplemental assistance categories 
are shown in Figure 4. 





Figure 4: Examples of Other Financial Assistance and 


Benefits 
Source of data: Ministry of Community and Social Services 


2008/09 







Maximum 
Amount ($) 


Expenditure 
Benefit Type and Frequency ($ million) 
community start-up and 

maintenance (once every 


24 months) 


special dietary allowance 
(monthly) 
employment-related 
expenses (monthly) 


799 single 
1,500 family 












other employment and 
employment assistance 
activities (once every 

12 months) 


funerals and burials 


253 






Examples of mandatory supplemental benefits 
for all eligible recipients or members of a recipient’s 
benefit unit (family) include community start-up 
and maintenance benefits, other employment and 
employment activities benefits, and dental and 
vision care benefits for dependent children. 

Other supplemental benefits can also be pro- 
vided on a case-by-case basis at the discretion of the 
service manager. Examples of discretionary benefits 
include dental and vision care for adults, the cost of 
funerals and burials, and moving expenses. 

Adequate documentation is to be placed in each 
recipient’s file or noted in the computer system to 
support the decision to pay the supplemental assist- 
ance and benefit, and the amount paid. 

Our review of the supporting documentation in 
a sample of recipient files found the following: 

e@ There often was no evidence on file that 
community or employment start-up events— 
such as moving or taking a course, for which 
one-time supplemental assistance was 
provided—had occurred. For example, one 
service manager automatically made an 
annual employment start-up payment of $253 
to everyone who was participating in any 
employment activity. However, the money is 
intended for recipients starting an activity for 
the first time that year, as opposed to a recur- 
ring annual payment. 

@ Similarly, the need for the various types of 
supplemental benefits provided often was not 
established. For example, several service man- 
agers automatically paid $100 a month for 
employment-related expenses to every par- 
ticipant that signed a participation agreement 
without establishing eligibility and requiring 
receipts. We noted that for one service man- 
ager, these monthly payments totalled more 
than $19 million in 2008. 

e In most cases, there were no receipts on file, 
nor were there any notes in the SDMT system, 
to demonstrate the reasonableness of the 
amounts paid to the recipients, contrary to the 
program’s requirements. 
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@ Payments for various types of supplemental 
benefits frequently exceeded the established 
maximums. For example: 

e Service managers frequently paid training 
fees in amounts ranging from $7,000 to 
$13,000 per year under the “employment- 
related expenses” category, which has 
an average annual limit of $3,000 and is 
intended for minor training costs, certifica- 
tion fees, and other costs that support a 
person’s progression to employment. This 
$3,000 limit can only be exceeded with 
formal approval by the service managers’ 
administrator, which was not received. 
Employment-related expenses are paid 
out of a set envelope for employment 
assistance. 

e In one instance, an individual received 
nearly $13,000 in overpayments because 
an entitlement had been incorrectly input- 
ted. Instead of receiving a one-time, $799 
payment for community start-up and main- 
tenance assistance, the person received 
that amount on a monthly basis for 17 
months before the error was discovered. 

We had similar observations relating to supple- 

mental payments in our 2002 Annual Report. 


Special Dietary Allowance 


A special allowance is to provide for additional 
assistance to each recipient and their families who 
require a special diet as a result of an approved 
medical condition. Before such an allowance can be 
provided, a health care professional—such as a doc- 
tor, nurse or dietitian—must complete an official 
application. A special-diet payment schedule issued 
by the Ministry is used to determine the amount of 
the allowance, depending on the medical condition. 
The amounts generally vary from $10 to $100 per 
condition per month. However, the total allowance 
for any one member of a family may not exceed 
$250 per month. 
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Province-wide, the total spent on special diet- 
ary allowances has increased substantially since 
the time of our last audit. In the 2002/03 fiscal 
year, annual special dietary payments totalled 
$5 million; in the 2008/09 fiscal year, the amount 
exceeded $67 million, a more than 12-fold increase. 
A significant part of this increase may be due toa 
campaign by advocacy groups critical of Ontario 
Works allowance amounts. At least one such organ- 
ization has organized clinics where health-care pro- 
fessionals have immediately completed special diet 
allowance applications that entitled each attendee 
to the maximum $250 monthly supplement. 

In light of the significant increase in special 
dietary allowance expenditures, one of the service 
managers that we visited took the initiative to 
review more than 1,000 of its clients receiving the 
allowance. It found that one of the 318 health-care 
practitioners who approved the 1,000 applications 
reviewed was responsible for approving almost 20% 
of them. As well, that same practitioner, a general 
practitioner, diagnosed, on average, nine medical 
conditions per applicant, compared to an average of 
about two per applicant diagnosed by other health- 
care professionals. Furthermore, this doctor diag- 
nosed Celiac disease in 99% of the applications, 
which we feel is unreasonably high given that the 
nationwide incidence of this disease is estimated 
at 1% of the population. This service manager, and 
one other that we visited, formally requested in 
2008 that the Ministry review the special dietary 
allowance province-wide. At the time of our audit, 
a formal province-wide review of the program had 
not been initiated. 

Our review of a sample of case files found the 
following: 

@ There were some instances where families 
consisting of eight to 10 members had all been 
diagnosed with the same multiple medical 
conditions, entitling all to the maximum 
special dietary allowance of $250 per month. 
As a result, some of these families were receiv- 
ing up to $30,000 a year from these dietary 





allowances alone, or approximately $50,000 
in total allowances annually, all tax-free. 

e@ There were a number of instances where an 
application for a special dietary allowance 
was completed by a health-care professional 
outside of the applicant’s immediate munici- 
pal area, which leads one to suspect that many 
applicants go to a professional that is known 
to be predisposed to approve such requests. 


RECOMMENDATION 4 





To ensure that supplemental financial assistance 
and benefits provided under the Ontario Works 
program are reasonable and appropriate, the 
Ministry of Community and Social Services 
should make certain that Consolidated Munici- 
pal Service Managers: 

@ comply with the requirement to document 
the need and eligibility for supplemental 
financial assistance and benefits, and pro- 
vide such assistance and benefits within the 
established maximum amounts; and 

@ obtain the required documentation to assess 
and substantiate the reasonableness of costs 
reimbursed. 

In addition, the Ministry should review the 
special dietary allowance with a view to limiting 
its possible abuse. 





The Ministry agrees that the supports provided. 
through Ontario Works should be reasonable 


and appropriate. The Ministry will reinforce 
with service managers the requirement to have 
appropriate documentation to support the 
provision of benefits within the established 
maximum amounts, where applicable. 

The Ministry is continuously looking for 
ways to improve the Special Diet Allowance. In 
2005, the Ministry introduced changes to the 
policy, the application process, and the applica- 
tion form in an effort to clarify the intent of 


the allowance and to reduce the potential for 
misuse. Since that time, the Ministry has been 
monitoring the allowance through consultation 
with municipalities and other stakeholders, and 
will reassess practices and procedures to iden- 
tify ways to further protect against misuse. 


Overpayments 


Overpayments occur when recipients are paid more 
assistance than they are entitled to receive. There 
are a variety of reasons for overpayments, ranging 
from fraudulent misrepresentation to incorrect 
evaluation of information. As of March 31, 2009, 
outstanding Ontario Works overpayments totalled 
$600 million. 

Specifically, overpayments to approximately 
60,000 active accounts totalled over $140 mil- 
lion, while overpayments to approximately 
290,000 inactive or terminated accounts totalled 
approximately $460 million. Of the total amount, 
$67 million had been declared as “temporarily 
uncollectible” and not subject to collection effort. 
The overpayment amounts do not include outstand- 
ing balances that were transferred to another social 
assistance program as a result of the recipient mov- 
ing to that program or that were written off. 

Collection efforts by municipal service managers 
from inactive or terminated accounts are extremely 
limited. In general, they consist of sending three 
SDMT-generated collection letters over a 60-day 
period requesting that the debtor make arrange- 
ments to repay the outstanding amount. Not 
surprisingly, the amounts collected by service man- 
agers from inactive or terminated accounts in the 
2008/09 fiscal year totalled less than $9 million, or 
2% of the outstanding $460 million. 

Service managers do not review or assess an 
overpaid individual’s ability to repay. As a result, 
they do not concentrate their collection efforts on 
former recipients who, for example, have returned 
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to work or have acquired considerable assets and 
may now have the ability to repay. 

Some service manager staff advised us that 
collection from inactive accounts is not seen as a 
priority because service managers pay 50% of the 
collection cost but retain only 20% of any collected 
amounts. In response to this issue, the Ministry 
initiated a pilot project with one service manager 
in 2006 whereby approximately $6.8 million from 
inactive, delinquent overpayment accounts was 
transferred to the Ministry’s Overpayment Recovery 
Unit for referral to the Canada Revenue Agency’s 
Refund Set-off Program. 

However, collections on these transferred 
accounts were also disappointing, and the pilot 
project has not been rolled out across the province. 
We note that one of the service managers we visited 
wanted to retain the services of an outside collec- 
tion agency but put that initiative on hold pending 
the outcome of the Ministry’s pilot project. 

With respect to collecting the $140 million 
in overpayments from active accounts, service 
managers can offset up to 10% of a recipient’s 
current monthly benefits against any outstanding 
overpayments. However, in practice, service man- 
agers generally limit the offset to 5%. In addition, 
service managers have designated about 10% of 
these active overpayment accounts as temporarily 
uncollectible for a variety of reasons, such as hard- 
ship to the individual. 

We found that, contrary to requirements, the 
reason for deferring collection of temporarily 
uncollectible accounts often was not documented in 
the system. In addition, caseworkers without super- 
visory approval can designate any overpayment 
as temporarily uncollectible and defer collection 
efforts indefinitely. 


RECOMMENDATION 5 . 


To better utilize its limited resources and maxi- 
mize the recovery of previous overpayments, 
the Ministry of Community and Social Services 
should: 
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@ ensure that Consolidated Municipal Service 
Managers assess the collectibility of all out- 
standing overpayments—particularly those 
designated as temporarily uncollectible— 
and, where warranted, recommend that 
the overpayments be written off so that 
more focus can be placed on those accounts 
where collection efforts are more apt to yield 
results; and 

e evaluate the merits of the 2006 pilot project 
that transferred some overpayments to the 
Ministry’s Overpayment Recovery Unit and, 
if necessary, consider implementing other 
alternatives for bringing a more intensive 
and focused collection effort to bear on those 
inactive accounts that have a greater likeli- 
hood of collection. 





The Ministry agrees that overpayment recovery 


must be maximized and has implemented busi- 
ness and technology changes to facilitate the 
recovery of overpayments. 

The Ministry is looking at the results of its 
2006 pilot project on overpayment recovery, 
and will continue with its efforts on mitigating 
overpayments, including the implementation of 
a risk-based approach to Ontario Works finan- 
cial eligibility reassessments. This risk model 
will help to ensure that only eligible recipients 
remain on the program, and that the payments 
they receive are in the correct amount. 

Finally, the Ministry is assessing the feasibil- 
ity of accelerating the write-off of aged overpay- 
ments and prioritizing overpayment collection 
on the basis of past and present recipients’ 
ability to repay. 


Potentially Fraudulent Claims 


Since the time our last audit, the Ministry revoked 
the lifetime ban for receiving Ontario Works 


benefits on applicants found to have made previ- 
ous fraudulent claims. The Ministry continues to 
operate a Welfare Fraud Hotline where people can 
report cases of suspected fraud. 

Currently, all fraud tips are to be assessed by 
service managers and, where appropriate, referred 
to a service manager’s eligibility review officer. 

If an investigation confirms that a recipient has 
received funds that the individual was not entitled 
to, income assistance is reduced or terminated, as 
appropriate. Where sufficient evidence exists to 
suspect intent to commit fraud, the case is to be 
referred to the police for investigation and possible 
criminal prosecution. 

We reviewed a number of tips received from 
the Welfare Fraud Hotline and had the following 
concerns: 

@ Two of the service managers we visited had 
no policies in place regarding timelines for 
investigations. As a result, many investigations 
into fraud tips were not considered in a timely 
manner. In fact, numerous tips had not been 
acted upon for up to three years. 

e@ In many cases, action taken was weak and 
inadequate. For example, at two service man- 
agers, Many recipients named in hotline tips 
were merely asked to sign a statement deny- 
ing the fraud allegation. 

e The number of cases that go forward to the 
police for fraud investigation is extremely 
low—approximately 1% of all tips at the 
service managers we visited. The service man- 
agers we visited indicated reluctance on their 
part, as well as on the police’s, to proceed with 
criminal action in most cases. 


_ RECOMMENDATION 6 


To ensure that only eligible individuals receive 
financial assistance and that adequate action 
is taken when suspected fraud is reported, the 
Ministry of Community and Social Services 
should ensure that Consolidated Municipal 
Service Managers: 


@ ina timely manner, follow up on all fraud 
tips and investigate those that appear to be 
legitimate; and 

e@ where the investigation indicates that a 
potential fraud has occurred, provide suf- 
ficient evidence to justice authorities to 
enable them to pursue prosecution of the 
perpetrators. 


The Ministry recognizes the need to ensure that 
only those who are eligible for social assistance 
receive it. Where sufficient evidence exists, 
social assistance staff are directed to refer all 
cases of suspected welfare fraud to the police. 
The Ministry will improve fraud investigation 
practices through the development of additional 
tools that support effective program manage- 
ment and oversight. 


Participation Agreements 


All Ontario Works assistance recipients must sign 
a participation agreement. The agreements oblige 
these individuals to take part in at least one of a 
number of activities designed to help transition 
them to paid employment and help them become 
self-reliant. Typical examples of employment activ- 
ities include: 

e basic education, literacy, and job-specific skill 
training; 

@ independent or assisted job-search activities, 
such as attendance at Employment Resource 
Centres; and 

@ volunteer or paid job placements designed 
to provide job experience and to help the 
recipient find and maintain meaningful 
employment. 

A service manager caseworker is to assess the 
individual’s skills and experience and determine 
with the recipient the most appropriate employ- 
ment activities. Every three months, participation 
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agreements are to be reviewed, updated, and 
signed again by the participant. 

However, we found little evidence in recipient 
files to indicate that caseworkers were actually 
assessing what training or other employment- 
directed activities would be most beneficial and 
there is no standard requirement to document this 
process. In fact, our understanding was that these 
activities were usually selected by the applicant. 
Our province-wide analysis indicated that two- 
thirds of all recipients had designated independent 
job-search activities as the most beneficial employ- 
ment assistance activity to help them become gain- 
fully employed. 

The requirement for an individual to enter 
into a participation agreement can be temporarily 
deferred in specific circumstances, such as a sole- 
support parent caring for one or more preschool 
children, provided the reasons for doing so are 
documented on file. 

Our review of a sample of files found that: 

@ Insome cases there were no participation 
agreements on file for extended periods of 
time. In many cases where participation 
agreements were on file, they were not 
updated every three months as required, 
and in some cases had not been updated for 
extended periods of time—up to five years. 

e@ Where an individual’s obligation to enter into 
a participation agreement was temporarily 
deferred, the reasons for doing so were often 
not documented in the file. 

e Every three months, caseworkers are required 
to monitor progress of the activities agreed to 
in the participation agreement with the recipi- 
ent. However, evidence was not required, nor 
was it provided, to demonstrate that the activ- 
ity to which the recipient had committed had 
ever been done. 

As well, we noted instances where individuals 
were in activities that seemed inappropriate. For 
example, individuals were in independent job 
search activities for several years, only to be sent 
later to English-as-a-second-language classes, 
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which in our view appears to be an ineffective 
sequence of activities. 

We also found many instances where it did not 
appear that recipients were getting an adequate 
assessment of what skills and experience they 
would need to secure employment. In one such 
case, the only training or assistance a recipient 
received was two short-term courses during a 
seven-year period. For the rest of the time, this indi- 
vidual was in an independent job search without 
any success. In addition, there was no evidence that 
this recipient had ever received a skills assessment. 
In another case, a 10-year recipient spent seven- 
and-a-half years without a participation agreement 
in place or any evidence that employment activities 
had taken place. For the rest of the time, the indi- 
vidual was in an independent job search or basic 
education program. 

In addition, although the Ministry does not 
define “temporary financial assistance,” which 
is the goal of the Ontario Works program, many 
individuals were in the program for long periods 
of time without progressing to financial independ- 
ence. In this regard, we noted that approximately 
10% of all active recipients at the three service 
managers we visited had been on continuous assist- 
ance for between five and 10 years, and an addi- 
tional 3% had been on for more than 10 years, with 
the oldest cases having received financial assistance 
since 1984, or for 25 years. 





RECOMMENDATION 7 





To ensure that the Ontario Works program 
is effective in transitioning recipients to paid 
employment and self-reliance, the Ministry of 

Community and Social Services should monitor 

Consolidated Municipal Service Managers to 

make certain: 

@ that participation agreements are on file for 
all Ontario Works recipients and that each 
agreement is reviewed and updated every 
three months as required; 


e that the reasons for deferring participation 
agreement requirements are adequately sup- 
ported and documented on file; 

@ that caseworkers assess recipients’ skills and 
experience, and document caseworker input 
in determining the most appropriate activ- 
ities to help recipients transition to financial 
independence; and 

e that the Ministry review the reasonableness 
of service managers’ allowing—often for 
prolonged periods of time—independent 
job-search activities as the primary employ- 
ment assistance activity to nearly two-thirds 
of all recipients. 





The Ministry agrees that all Ontario Works 


recipients should have a participation agree- 
ment on file, which will be reviewed and 
updated at least every three months, and that 
any deferrals of participation agreements are 
supported and documented on file. 

Recognizing the diverse challenges that 
many Ontario Works recipients face, the 
Ministry is committed to providing tools and 
training support to help front-line staff work 
collaboratively with clients to address their 
employment-related needs and barriers. The 
Ministry will review the policy guidelines 
related to the job-search requirement to ensure 
that participation agreements are developed or 
updated appropriately. 


Tasks 


The Ministry's SDMT system is able to assign tasks 
and corresponding due dates to individual case 
files as well as track outstanding tasks. Tasks are 
system-generated for such things as notification 
that a recipient’s supplemental assistance or benefit 
is about to expire, or the need to review and update 
participation agreements. Many of the remaining 


tasks are entered manually and are triggered by, for 
example, a complaint about a person’s eligibility or 
information obtained from third parties through 
information-sharing agreements. 

It is essential that caseworkers be aware of all 
tasks as they become due so that any necessary 
changes can be made promptly and overpayments 
or underpayments can be avoided. However, at 
the time of our audit, there were 195,000 overdue 
Ontario Works tasks in the SDMT system. Many of 
these tasks had been overdue for a long time, some 
for more than 10 years. 

We understand that the high number of overdue 
tasks is due largely to service manager staff not 
using the system as intended to identify informa- 
tion needs, by staff not obtaining the required 
information on a timely basis, or by failing to delete 
tasks when they are completed. This large number 
of overdue tasks is of particular concern because 
they are the key means of consistently tracking 
outstanding information needed to establish the 
continuing eligibility of recipients and determining 
the correct amount of assistance to be paid. 


RECOMMENDATION 8 
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To ensure that Ontario Works benefits continue 
to be paid only to eligible individuals and in the 
correct amount, the Ministry of Community 
and Social Services should monitor whether 
Consolidated Municipal Service Managers are 
making reasonable efforts to address all system- 
identified tasks that require action or follow up. 


The Ministry recognizes the need to ensure 
that only eligible individuals receive assistance 
through Ontario Works and that the assistance 
provided is in the correct amount. To this end, 
the Ministry has simplified the technology 
related to system-generated tasks. 

The Ministry is also reviewing its current 
business processes for potential refinements 
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and opportunities for improvement from the 
perspective of technology modernization. 


Ministry Monitoring of Consolidated 
Municipal Service Managers 


The Ministry’s nine regional offices are to regularly 
conduct two types of reviews—compliance reviews 
and subsidy claims examinations—of the service 
managers within their jurisdiction. Compliance 
reviews consist primarily of examining a sample of 
case files to assess whether they adhere to selected 
program requirements. These reviews are currently 
to be conducted on a three-year cycle. 

Subsidy claims examinations inspect a single 
month’s reimbursement claim by the service 
manager for the Ministry’s 80% share of financial 
assistance provided to recipients. These examina- 
tions, to be conducted annually, are to ensure that 
the amounts reimbursed to service managers accur- 
ately reflect payments to recipients. 

When it came to compliance reviews, we found 
that they were being conducted at the required 
frequency. That is, annually in 2003, once every 
two years between 2004 and 2006, and once every 
three years starting in 2007. Our review of a sample 
of compliance reviews since our last audit found 
that the work undertaken was generally of a good 
quality and identified many of the same issues and 
concerns we have drawn attention to earlier in this 
report. However, there was little evidence that cor- 
rective action to address the deficiencies identified 
in the compliance reviews was undertaken, as simi- 
lar issues kept recurring from year to year. 

In response, the Ministry implemented a new, 
three-year cycle for compliance reviews starting in 
2007. The second year is to allow the service man- 
ager time to take the necessary corrective actions 
identified during the year-one review. In the third 
year, the Ministry is to re-examine a sample of case 
files and assess whether or not corrective actions 
have been taken. Financial adjustments may be 
levied for issues of non-compliance. 
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With respect to subsidy claims examinations, we 

noted the following: 

e The Ministry’s regional offices were not com- 
pleting subsidy claims examinations annually 
or on a timely basis as required. In one of the 
three visited regions, for example, reviews for 
2003 were conducted in 2006 and reviews 
for 2004 and 2005 were conducted in 2008. 
In the other two regions, reviews for 2003, 
2004, and 2005 were not completed at all. No 
reviews had been done at any of the regions 
for 2007 and 2008. On a province-wide basis, 
as of December 2008, the 2007 subsidy claims 
examinations had been completed for only 
four of the 47 service managers. 

@ Our review also found that, generally, the 
reviews were inadequately conducted. Files 
were disorganized, difficult to follow and 
incomplete. In addition, it is our view that 
many of the individuals conducting the 
subsidy claims examinations did not have 
adequate training and experience to do so 
effectively. As a result, we felt that the subsidy 
claims reviews did not adequately determine 
whether the claims submitted to, and paid 
by, the Ministry were complete, accurate, 
and based on actual benefits provided to 
recipients. 

These reviews are critical given the fact that 

the Ministry’s subsidies totalled $1.5 billion in the 
2008/09 fiscal year. Yet none of the required sup- 
porting documentation that is required to accom- 
pany the monthly claims was being submitted to 
the Ministry. As well, the Ministry did not verify 
any of the information on the claims prior to mak- 
ing payment. The risk is that if a service manager 
inadvertently overstated a claim, the error likely 
would not be detected. 


| RECOMMENDATION 9 


To ensure that subsidy claims are reimbursed in 
the correct amount based on reliable informa- 
tion provided by the Consolidated Municipal 


Service Managers, the Ministry of Community 
and Social Services should: 

e@ conduct at least one subsidy claims exam- 
ination per service manager annually as 
required and do so on a timely basis; 

e make certain that work conducted during 
subsidy claims examinations is adequately 
completed and demonstrates whether the 
claim is based on complete and accurate 
information about payments to assistance 
recipients; and 

e make certain that adequate supporting 
documentation is submitted by the service 
managers and reviewed by the Ministry prior 
to payment. 


The Ministry recognizes the importance of 
exercising appropriate program management 
oversight, and has reinforced the requirement to 
complete annual subsidy claims examinations. 
In addition, the Ministry will develop additional 
tools and provide training to support ministry 
staff in completing accurate examinations based 
on appropriate documentation. 


Program Administration Costs 


At the time of our last audit in 2002, we found that 
the Ministry reimbursed the 47 service managers 
for their 50% share of administrative costs based 
on a historical pattern that ignores, among other 
things, caseload volumes. We determined that the 
Ministry’s administration cost reimbursement on 

a per-case basis in the 2001/02 fiscal year varied 
significantly, ranging from $273 to $1,596. We 
therefore recommended in our 2002 Annual Report 
that the Ministry consider caseload information in 
its annual funding decisions to ensure that admin- 
istration costs are allocated equitably across the 
province. The Ministry agreed and indicated at the 


time that future funding for its share of administra- 
tion costs would be linked to caseloads. 

Notwithstanding that commitment, we found 

that the Ministry continues to reimburse service 
managers on the same historical basis established 
prior to our last audit in 2002. The Ministry’s fund- 
ing of service manager administration costs on a per- 
case basis continued to vary significantly between 
$718 and $1,250 in the 2008/09 fiscal year. 

We also noted the following: 

@ The Ministry does not receive sufficiently 
detailed information about the administration 
costs incurred by individual service managers 
and therefore cannot assess their reasonable- 
ness. In addition, due to the absence of a prov- 
incial requirement of caseloads per caseworker, 
it is impossible to assess the appropriateness of 
caseworker staffing levels. This is particularly 
important since staffing accounts for approxi- 
mately 80% of all administration costs. 

@ Some service managers are absorbing all the 
incremental costs, such as salary increases, for 
administration of the program, which results 
in cost sharing that differs from the intended 
50/50 basis. For example, one of the service 
managers that we visited estimated that it 
currently pays 70% of the total administration 
costs. As a result, service managers unable 
to absorb the incremental costs of program 
administration could, for example, decrease 
caseworker staffing levels, which would 
adversely affect program delivery. 


RECOMMENDATION 10 


To ensure that Ontario Works administration is 





funded equitably across the province, the Min- 

istry of Community and Social Services should: 

@ establish more needs-based funding of 
administrative costs that reflects variations 
in caseloads; and 

e obtain better information about actual 
administrative costs being incurred. 
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The Ministry recognizes the concerns with pro- 
gram administration funding and is currently 
undertaking a review to develop principles for 
revising the funding model. 


Employment Assistance Costs 


As noted previously, the Ministry’s 80% share of 
employment assistance costs totalled $171 million 
in the 2008/09 fiscal year. Much of this assist- 
ance is provided directly by service manager staff, 
although some services are obtained through 
contractual arrangements with third parties such 
as providers of training programs and employment 
placement services. Employment assistance fund- 
ing provided to individual service managers is still 
generally based on historically funded amounts 
rather than an assessment of recipient caseloads 
and the need for the different types of employment 
assistance services. However, the Ministry advised 
us that it started to implement an outcome-based 
funding model in January 2008 that will begin to 
affect funding allocations in 2010. 
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Our comments and concerns with respect to 





employment assistance funding provided to indi- 
vidual service managers over the past few years 
include the following: 

@ There is no evidence that the Ministry 
assessed the type and mix of employment 
activities provided by a service manager 
to ensure that they are effective in helping 
transition assistance recipients from Ontario 
Works to paid employment and ultimately 
represent value for money spent. In that 
regard, we note that two-thirds of all assist- 
ance recipients are receiving no specific 
employment assistance and are assigned to 
independent job search activities, often for 
many years. 

e@ The Ministry did not receive sufficiently 
detailed information on how the employment 
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assistance funds were to be spent and were 
actually spent. In fact, we found that in some 
cases service managers used employment 
assistance funds for other Ontario Works pur- 
poses or for unrelated municipal programs. 

e There was often no evidence that municipal 
service managers acquired employment 
assistance services from third-party providers 
competitively. 


RECOMMENDATION 11 


To ensure that employment services are effect- 
ive in helping recipients find employment and 
represent value for money spent, the Ministry of 
Community and Social Services should: 

e assess the effectiveness of the various types 
of employment assistance being offered by 
each Consolidated Municipal Service Man- 
ager, particularly the independent job search 
when recipients are assigned to it for long 
periods of time; and 

@ make certain that all employment assist- 
ance funding is spent prudently and for the 
intended purpose. 





The Ministry introduced an outcomes-based 


funding model for Ontario Works employment 
assistance that requires service managers to 
establish performance targets and measure 
client outcomes. 

The Ministry recognizes the importance of 
exercising appropriate program management 
oversight and will ensure that ministry staff 
receive training to support effective oversight of 
employment assistance funding. In addition, the 
Ministry is currently looking at the employment 
assistance funding model as part of its review of 
administration funding. 





Measuring the Performance of the 
Ontario Works Program and Consolidated 
Municipal Service Managers 


Historically, the Ministry has lacked any measures 
to monitor and evaluate the efficiency and effect- 
iveness of the administration of income assistance 
under Ontario Works. For example, targets have 
not been established with respect to the reduction 
and/or elimination of income assistance overpay- 
ments to recipients, even though income assistance 
is by far the largest (81%) cost component of the 
$1.9-billion program. 

In addition, there is no question that it is chal- 
lenging to evaluate effectiveness in achieving the 
primary objective of the program—to move Ontario 
Works recipients to paid employment and self- 
reliance—because many factors not related to the 
program can influence the number of people leav- 
ing it. These include, but are not limited to: 

® conditions in the general economy that 
greatly influence the creation or loss of the 
types of jobs Ontario Works recipients are 
most likely to qualify for; 

e local conditions and seasonal factors that 
influence the availability of jobs in a given 
area; and 

e the commitment and personal initiative 
of Ontario Works recipients to find paid 
employment. 

With respect to the employment assistance 
component of Ontario Works, the Ministry in 2008 
started to implement a new outcomes-based model 
that will measure performance over a two-year per- 
iod. This model includes seven outcome measures 
in two categories: earnings outcome and employ- 
ment outcome. Under earnings outcome, there are 
two measures: average employment earnings for 
Ontario Works recipients and average employment 
earnings at exit from the program. 

Under employment outcome, there are five 
measures: 

@ average length of time in the program until 

exit to employment; 


@ percentage of caseload terminations as a 
result of exiting to employment; 

@ percentage of caseloads with some employ- 
ment income; 

@ job retention rate—average length of time 
those people who had been in Ontario Works 
held a job before returning to the program; 
and 

@ re-entry rate—percentage of people returning 
to Ontario Works who had left the program 
for employment within the past 24 months. 

Each year, the Ministry negotiates improvement 

targets with each municipal service manager for 
the above outcome measures based on historical 
patterns and local economic conditions. Evalua- 
tion is to take place over a two-year period. Over- 
achievement in year one of the two-year evaluation 
cycle (that is, initially, 2008) can be used to offset 
underachievement in the second year or vice versa. 
Underachievement over the initial two-year evalua- 
tion period, which ends in December 2009, may 
result in the Ministry clawing back up to 20% of a 
service manager’s employment assistance funding 
that it received during that two-year period. 

This is a promising initiative if the Ministry 

can obtain complete and accurate information 
regarding the seven outcome measures. 


RECOMMENDATION 12 


The Ministry of Community and Social 

Services should build on its planned results- 
assessment for employment assistance funding 
by developing performance measures that will 
enable it to evaluate the effectiveness of the 
administration of the much larger income assist- 
ance aspect of Ontario Works over time. 


The Ministry recognizes the importance of 
exercising appropriate program management 
oversight with respect to income assistance. 
The Ministry will continue with its efforts to 
maximize overpayment recovery and mitigate 
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overpayments, including the implementation of 
a risk-based approach to Ontario Works finan- 
cial eligibility reassessments. This risk model 
will help to ensure that only eligible recipients 
remain on the program and that they receive 
the correct payments. 

The Ministry will also implement a series 
of changes to reinforce its monitoring and con- 
trollership framework for social assistance. This 
framework will include performance-monitoring 
and risk-management strategies that will 
strengthen program oversight and support the 
improvements being made to service delivery. 


SERVICE DELIVERY MODEL 
TECHNOLOGY SYSTEM 


The Ministry’s Service Delivery Model Technol- 
ogy (SDMT) information system is the IT network 
that supports social assistance delivery for both 
Ontario Works and the Ontario Disability Support 
Program. Implemented province-wide in 2002, the 
SDMT system was developed to provide a common 
database with real-time access to case information 
and to reduce administrative costs while freeing up 
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caseworker time to allow for better customer ser- 
vice to social assistance applicants and recipients. 

The SDMT system was developed at an initial 
cost of approximately $246 million, which far 
exceeded the original cost estimate of $180 million. 
The Ministry now estimates the total cost to date 
for system development and maintenance paid to 
outside contractors at approximately $377 million. 
The Ministry took control of this system in Janu- 
ary 2002. Since then, the SDMT system has been 
maintained by in-house ministry staff, supported by 
outside consultants. The cost for these outside con- 
sultants was approximately $5.5 million between 
the 2005/06 and 2008/09 fiscal years. 

Our review of the SDMT system included a sur- 
vey of users and administrators to determine satis- 
faction and areas of concern. We also engaged an IT 
security specialist to conduct a security review. 
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Although the Ministry has made many changes 
to the SDMT system over the years to improve the 
consistency and accuracy of the system’s operations, 
much remains to be done. We understand that the 
government has designated the SDMT system as a 
priority as part of its project to remediate high-risk 
applications. The Ministry is reviewing its current 
business processes for potential refinements or 
opportunities to improve the system’s technology. It 
is to prepare a business case for autumn 2009. 

Our specific comments and observations about 
the SDMT system are detailed in the following 
section. 


Unexplained System Errors and Omissions 


We again found—as we did in our 2002, 2004, 
and 2006 annual reports—that unexplained errors 
and omissions continue to occur, even after many 
system enhancements. For instance: 

e Some changes made to a recipient’s informa- 
tion in the SDMT system were not immedi- 
ately processed and were dormant for months 
and even years. Then, at a much later date, 
they were triggered for unexplained reasons. 
Such a situation may result in significant 
arrears or overpayments. For example, in one 
case a recipient incurred decreased shelter 
costs in 2002 that were entered into the SDMT 
system, but not used in determining the 
correct amount of financial assistance. The 
recipient continued to receive the previous, 
larger shelter allowance for seven years until 
the system detected the overpayment. The 
service manager could not explain why this 
error occurred. 

e Information regarding the same payments 
made to recipients during a particular month 
showed different amounts that were con- 
tained in two monthly SDMT system reports— 
the expenditure report and the cheque 
register, which is a list of cheques generated 
by the system during that month. Neither the 


service managers nor the Ministry were able 
to explain the discrepancies. 

@ The SDMT system lacks controls to detect 
input errors and omissions. For example, the 
system does not have the capacity to block 
payments to recipients in cases where a 
unique personal identifier—social insurance 
or health card number—has not been input- 
ted. We found many cases where these unique 
identifiers were missing, sometimes for 
more than a year, while recipients continued 
to receive assistance. This system failure 
increases the risk of fraud through multiple 
payments to the same recipient or payments 
to false recipients. 


Access and Security Controls 


We are pleased to report that attempts to gain 
unauthorized access to the SDMT system met with 
failure during a security test, which suggests that 
there is a reasonable level of security control to 
protect the system from possible outside attacks. 
However, we are concerned about internal access 
and overall system controls to prevent the SDMT 
system from being compromised. In this regard, we 
found the following: 

e According to our security specialist, the 
possibility exists for an internal user with IT 
knowledge to escalate their read-only access 
to full access to SDMT data without proper 
authorization. This would allow an indi- 
vidual to create a bogus recipient and issue 
fraudulent payments. The Ministry had been 
aware of these issues and thought that it had 
corrected them, but our specialist was able to 
circumvent the new controls. 

e With regard to access rights, we found that 
although only two staff members per service 
manager office were supposed to be provided 
administration rights, which includes the abil- 
ity to make changes and generate new users, 
some Offices had up to 17 individuals, or one- 
third of their total staff, assigned these rights. 


Administrators were also provided access to 
live data that should be the purview only of 
caseworkers. As a result, administrators pot- 
entially can establish false accounts and new 
users, generating unauthorized payments. 

e Although the SDMT system has the capacity 
to assign limited access levels, we found that 
most caseworkers, as we noted in our 2002 
audit, received full access to the system, 
allowing them also to set up new recipients, 
make changes to information, and potentially 
authorize fraudulent payments, all without 
supervisory review. Such broad access runs 
counter to the desired segregation of duties 
and supervisory oversight that is a critical 
component of a formal payment system 
designed to prevent fraudulent payments. 

@ Although the Ministry had a process in place 
to verify active users by sending a SDMT 
system report containing all active users to 
service managers for their review and recon- 
ciliation, this feature had not been used since 
2005. In fact, our review found that some for- 
mer Ontario Works staff still had active SDMT 
system accounts. 

@ Passwords are not required to be changed ona 
regular basis and multiple concurrent log-ins 
are permitted. 


User Satisfaction 


During our previous audits, service manager staff 
expressed considerable dissatisfaction with the 
SDMT system and told us that instead of it freeing 
up time to spend with clients as intended, it had 

the opposite effect. Other concerns noted were that 
training was insufficient, and that system limitations 
required many workaround systems to be developed 
or purchased in order to get the job done. 

As mentioned above, despite some SDMT system 
improvements since our last audit, service manager 
staff still generally express dissatisfaction with the 
system. They had the following specific concerns: 
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e Workaround systems are still required. Results 


from the survey of system administrators 
indicated that approximately 150 workaround 
systems had been developed at a cost of more 
than $5 million, with future system develop- 
ment costs estimated at $7 million. Many 

of these workarounds were similar systems 
developed by different service managers, 
resulting in a duplication of efforts. 

The SDMT system lacks a report-writing 
function that would allow service managers 
to extract customized information required to 
assist with program delivery and/or manage- 
ment. Instead, the Ministry provides daily and 
monthly information for use in local report 
systems as well as producing standard reports 
that are available to the service managers. 
However, these reports do not address many 
of their information needs. Although special 
reports can be requested from the Ministry, 
service manager staff told us that it sometimes 
takes several months to receive these reports. 
In addition, service manager staff were reluc- 
tant to rely on the reports because of concerns 
over reliability, completeness, and accuracy. 
Although a process had been set up to flag sys- 
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tem problems by filing a system investigation 





report, most service manager staff felt that 
this process was ineffective and did not result 
in improvements. The Ministry has since elim- 
inated this process and no longer tracks SDMT 
system user complaints. 

The Ministry now uses the government-wide 
IT service desk to deal with SDMT system 
problems. Staff told us that they have con- 
cerns about the quality of assistance they 
receive from this help desk as those staffing it 
don’t seem to have specific knowledge about 
the SDMT system and are not responsive in 
addressing problems. 


e Although tools are provided to assist with 


reviewing overpayments, the system’s users 
noted the system was unable to determine 
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why an overpayment was created, an issue 
previously noted in our 2002 audit. They 
indicated that it sometimes took a long time 
to try to resolve and reconcile overpayments. 
In addition, the SDMT system lacks the 
capability to manage overpayments and their 
collections. 

e The system lacks the capacity to manage the 
employment assistance function, a key object- 
ive of Ontario Works. As a result, most service 
managers maintain standalone systems to 
support the management of employment 
assistance activities. 

e Concerns were noted again, as at the time 
of our last audit, with regard to system- 
generated letters. Service manager staff said 
that these letters cannot be altered, that the 
information contained in them is difficult for 
the client to understand, and is sometimes 
inaccurate. As a result, many service man- 
agers have purchased or developed other soft- 
ware programs to create their own letters and 
do not use this aspect of the SDMT system. 


Information to Support Reimbursement by 
the Ministry 


In our previous report in 2002, we noted that the 
SDMT system did not provide municipal service 
managers with accurate and reliable expenditure 
information for billing the Ministry for its share of 
the financial assistance provided to Ontario Works 
recipients. We are pleased to note that now, in 
general, service managers are able to rely on the 
information from the SDMT system with regard to 


the income assistance amounts provided to Ontario 
Works recipients. 

However, benefits that are paid on behalf of 
the recipients to third parties are not included 
in the SDMT system and have to be manually 
added onto the monthly claim to the Ministry for 
reimbursement. 

As previously noted, small variances still 
exist between the totals recorded by the monthly 
expenditure report and the cheque register, both 
produced by the SDMT system. 


") RECOMMENDATION 13 





To ensure that Consolidated Municipal Service 
Managers can rely on systems and reports 

to produce proper payments, and accurately 
record and manage information regarding 
those payments, the Ministry of Community 
and Social Services should address the Service 
Delivery Model Technology system deficien- 
cies noted in this report, including those that 
prevent service manager staff from having the 
information they need to effectively manage 
program expenditures. 





The Ministry recognizes the need to continually 


improve the technology that supports the deliv- 
ery of the Ontario Works program within avail- 
able resources. The Ministry is also reviewing its 
current business processes for potential refine- 
ments and opportunities for improvement from 
the perspective of technology modernization. 







Section 


Background 


Affordable and secure housing is a significant factor 
in a community’s stability and in the social and eco- 
nomic well-being of its residents. Social housing is 
rental accommodation developed with government 
assistance for a range of low- and moderate-income 
households, including families with children, 
couples, singles, and seniors. It can be owned by 
governments, as in the case of public housing, or by 
non-profit or co-operative organizations. In Ontario, 
households in social housing that receive a rent- 
geared-to-income subsidy typically pay a maximum 
rent equal to 30% of their total pre-tax income. 

Most social housing in Ontario was built 
between the mid-1960s and the mid-1990s through 
a combination of federal, provincial- and joint 
federal—provincial cost-shared programs. Commun- 
ity groups also built non-profit and co-operative 
housing during the 1980s and 1990s, with more 
emphasis on smaller projects that included units 
with rents at market rates alongside those with rent 
geared to income. 

Until the late 1990s, properties built by govern- 
ments (that is, public housing properties) were sub- 
sidized and administered by Canada Mortgage and 
Housing Corporation (CMHC) at the federal level 
and Ontario Housing Corporation at the provincial 
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level. The non-profit and co-operative housing built 
by community groups was administered by those 
groups but funded by both levels of government. 

In January 1998, the province transferred to 
municipalities a series of funding responsibilities, 
including social housing, under the Local Services 
Realignment program. In return, the province 
assumed half the education costs that had previ- 
ously been paid by municipalities. The Ministry of 
Municipal Affairs and Housing retained responsibil- 
ity for administering social housing programs dur- 
ing a transition period. 

In November 1999, the federal government and 
Ontario signed the Canada—Ontario Social Hous- 
ing Agreement (Agreement) to transfer federal 
administration of most Ontario social housing 
to the province (although CMHC continued to 
administer certain housing programs). The Agree- 
ment provides the province with the flexibility to 
devolve administration of social housing programs 
to municipalities, set policies for client assistance, 
and allocate federal funding to the various housing 
programs. The province continues to receive federal 
funds ($518 million in the 2008 calendar year), 
most of which it allocates to municipalities. 

In December 2000, the province passed the 
Social Housing Reform Act, 2000 (Act), which 
required municipalities to assume responsibility for 
social housing programs previously administered 
by both CMHC and the province. The province 
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designated 47 regional Consolidated Municipal 
Service Managers (Service Managers), who also 
deliver such other services as social assistance and 
child care, to administer social housing programs 
at the local level. In bigger cities, the municipal 
government itself is the Service Manager; in smaller 
centres, a single Service Manager administers 
services for a combined group of municipalities 
and counties. Ontario is the only province to have 
passed on to municipalities the responsibility for 
funding and administering social housing. 

As of the end of 2008, there were about 260,000 
units of social housing in Ontario, consisting of 
100,000 public-housing units and 160,000 non- 
profit and co-operative units. Although no formal 
figures were available, the asset value of the 
province’s social housing stock was estimated to be 
approximately $40 billion. 





Audit Objective and Scope 


The objective of our audit was to assess whether 
the Ministry of Municipal Affairs and Housing 
(Ministry) had adequate systems and procedures in 
place to: 
® measure and report on its effectiveness in 
helping to provide, in partnership with the 
federal and municipal governments, sufficient 
numbers of well-maintained social housing 
units; and 
e ensure that funds provided for selected hous- 
ing programs are managed with due regard 
for economy and efficiency, and in compliance 
with legislative and program requirements. 
Our audit included research into the practices of 
other jurisdictions, a review of documentation, and 
interviews with ministry staff and some of the 47 
Consolidated Municipal Service Managers (Service 
Managers) to obtain their views on the delivery of 
social housing programs. We also sent surveys to 
all 47 Service Managers and received responses 
from about half of them. We also interviewed 


external stakeholders, including the Association of 
Municipalities of Ontario, Social Housing Services 
Corporation, the Ontario Non-Profit Housing Asso- 
ciation, Toronto Community Housing Corporation, 
and Canada Mortgage and Housing Corporation 
for their input on how housing programs could be 
improved. 

Our work also included a review of relevant 
audit reports issued by the Ministry’s internal aud- 
itors. Since they had not conducted any recent work 
in the areas covered by our audit, their reports did 
not result in a reduction of the scope of our audit or 
extent of our procedures. — 


From both a value-for-money perspective and 

from the perspective of those who live in it, it is 
critical that social housing be maintained in good 
condition. As well, sufficient and affordable social 
housing can have a significant impact on the health 
and safety of those Ontarians who depend on 
subsidized housing for a place to call home. While 
responsibility for this has been largely delegated by 
legislation to municipalities since 2000, it is in the 
province’s long-term interests to monitor how well 
the province’s social housing stock is being man- 
aged by Consolidated Municipal Service Managers 
(Service Managers). 

However, despite the change in responsibilities, 
there has been no provincial strategy to help ensure 
long-term sustainability of sufficient numbers of 
well-maintained social housing units. Accordingly, 
other than ensuring that any federal or provincial 
housing agreements and other requirements are 
being adhered to, the Ministry collects little infor- 
mation on how well the $40 billion in social hous- 
ing stock is being maintained or whether there is an 
adequate supply to meet the local needs. We identi- 
fied a number of issues about which we believe the 
province should be better informed to enable it to 
monitor how well social housing is being managed, 
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especially given the province’s recent commitment maintain the prescribed minimum number 

to provide municipalities with more than $600 mil- of housing units. The Ministry had no firm 

lion (its half of the $1.2 billion federal—provincial plans to address Service Managers’ concerns 

economic stimulus package provided to it) for new regarding the possible ending of federal 

and existing housing programs. Some of these funding. 

issues include: e In partnership with the federal government, 
@ As of December 2008, the number of house- Ontario has in recent years provided Service 


holds on waiting lists for social housing across 
the province totalled about 137,000. In many 
urban centres, the average wait time to secure 
accommodation was more than five years— 
and one municipality had reported a wait time 
of 21 years for all categories except seniors. 
Some large municipalities reported that as 
units became available, 25% to 40% were 
usually allocated to special-priority tenants, 
such as victims of abuse, who require complex 
social-support services. However, housing 
providers often do not have the capacity to 
provide security and complex support to such 
special needs tenants, nor were security and 
services for such tenants well co-ordinated 
with other programs. 

The deteriorating condition of the social 
housing stock, particularly the public-housing 
portfolio, whose units are an average of 40 
years old, has been a significant and grow- 
ing concern for municipalities. In 2006, for 
instance, the Toronto Community Housing 
Corporation conducted a building-condition 
assessment, which identified immediate cap- 
ital-repair needs of $300 million for its 60,000 
social housing units. However, the Ministry 
had no up-to-date and reliable information on 
a province-wide basis of the overall condition 
of the social housing stock or of the asset- 
management practices of Service Managers. 
A large number of the federal government's 
funding agreements with housing providers 
will start to expire in 2015, with no guarantee 
they will be renewed. Without continued 
funding, some existing social housing projects 
will not be financially viable. However, Ser- 
vice Managers will still be required by law to 


Managers with some additional funding for 
new housing programs. Although the Ministry 
monitors whether Service Managers comply 
with program requirements, there was a gen- 
eral lack of reporting on the success of these 
programs. We determined, for example, that 
although one such program did increase the 
supply of housing, the stipulated rent to be 
charged meant that more than half the units 
would not be considered affordable for house- 
holds on waiting lists, or those eligible to be 
on the lists. 

e As part of a federal-provincial economic 
stimulus package, both levels of government 
announced in March 2009 they would share 
equally in funding $1.2 billion in new invest- 
ments in social and affordable housing over 
the next two years. Improvements to the Min- 
istry’s system for monitoring these expendi- 
tures will be needed to ensure these funds are 
spent cost-effectively and achieve the desired 
impact. 


The Ministry of Municipal Affairs and Housing 
welcomes the observations and recommenda- 
tions of the Auditor General and is committed to 
making continuous improvement in meeting its 
mandate. The Ministry recognizes the integral 
role of safe and affordable housing in the health 
and safety of Ontarians and is focused on sup- 
porting our municipal partners as they adminis- 
ter social housing in Ontario. 

Since devolution in 2001, all housing 
partners, including the province, have been 
maturing in their new roles in administering 
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social housing in Ontario. The recent Provincial 
Municipal Fiscal and Service Delivery Review 
confirmed that the province would not upload 
responsibility for social housing from the Ser- 
vice Managers. The review recommendations 
supported continued municipal funding and 
administration, while the province will provide 
the opportunities to free up additional funds for 
social housing. The role of the province to over- 
see the success of the program has also evolved 
and the Ministry is committed to effectively 
meeting this role. : 


Detailed Observations 





COMPLIANCE WITH AGREEMENT AND 
LEGISLATION 


The Canada—Ontario Social Housing Agreement 
(Agreement) with the federal government estab- 
lishes certain reporting requirements that the Min- 
istry of Municipal Affairs and Housing (Ministry) 
is required to follow. Specifically, the Ministry 
must submit an annual performance report to the 
federal government and undertake annual audits 
and periodic program evaluations. The annual per- 
formance report must include information on the 
amount of funding for each program, the number 
of households assisted, and the average income of 
those receiving assistance. 

In addition, under the provincial Social Housing 
Reform Act (Act), Consolidated Municipal Service 
Managers (Service Managers) are required to fulfill 
a number of key responsibilities that include: 

® maintaining the prescribed minimum number 

of housing units; 

e establishing and administering waiting lists; 

@ providing rules about eligibility and prior- 

ity rules, creating occupancy standards, 
and ensuring adherence to these rules and 
standards; 


@ reporting on projects in difficulty; 

e providing subsidies to, and oversight of, hous- 

ing providers; and 

® reporting on compliance with social housing 

obligations to the province. 

To help ensure compliance with the federal and 
provincial requirements, the Ministry requires Ser- 
vice Managers to submit a Service Manager Annual 
Information Return (Return). Service Managers 
must state in the Return the expenditures incurred 
for each housing program, provide certain supple- 
mental information, and confirm their compliance 
in key areas such as maintaining the prescribed 
number of housing units and a centralized waiting 
list. An external auditor hired by the Ministry must 
verify Returns and issue an audit opinion to CMHC 
on the federal money spent. 

We found that the Returns were being veri- 
fied and, based on the opinion of the ministry- 
appointed auditor, the province and municipalities 
were in compliance overall with the Agreement and 
legislation. 


PROVINCIAL STRATEGY ON SOCIAL 
HOUSING 


Service Managers currently have primary respon- 
sibilities for funding and administering social hous- 
ing programs. Even so, social housing is a shared 
responsibility and the Ministry, in partnership with 
the federal and municipal governments, is account- 
able to Ontarians for providing sufficient and well- 
maintained social housing across the province. 

Despite the significant change in the responsibil- 
ities for delivery of social housing, however, there 
had been no provincial strategy to address potential 
issues that could affect the provision of sufficient 
and well-maintained housing in the province. 
Beyond the annual compliance-reporting process, 
the Ministry had not adequately overseen the suc- 
cess of municipal service delivery. 

A provincial strategy is needed to define the 
Ministry’s roles, set measurable goals and program 
priorities, assess risks and options to manage 


the risks, determine the resources required, and 
measure the impact of the Ministry’s contribution 
to social housing. In the Ministry’s recent Results- 
based Plan (Plan), a document that all Ontario 
government ministries are required to submit to 
help ensure their programs achieve the desired out- 
comes, the Ministry had identified several goals and 
the various activities on which it intended to report. 
However, we found these goals and activities as 
reported to be overly broad, with no measurable 
outcomes. For example, one of the activities identi- 
fied was the maintenance and upgrading of aging 
social housing units—but the Plan established no 
targets or benchmarks for success. 

The need for a provincial strategy was under- 
scored in May 2006, when a consultant engaged by 
the Ministry to conduct an evaluation of its social 
housing programs noted a number of issues similar 
to the ones we observed. The consultant’s evalua- 
tion, required under the Canada—Ontario Social 
Housing Agreement, noted, among others, the fol- 
lowing issues: 

e a lack of strategic performance measurement 

across Ontario’s social housing programs; 

@ the absence of province-wide benchmarks, 

metrics, and objectives for social housing; and 

e a lack of strategic-planning initiatives to 

address issues, including emerging capital 
requirements, increasing operating costs, and 
demographic and economic changes that may 
affect long-term sustainability. 

It has been three years since these issues were 
identified but little action has been taken to date to 
address them. 

In this regard, a number of municipalities 
had developed a local strategic plan to address 
the issues within their communities. Our review 
indicated some of these plans were comprehensive 
and could be useful to the Ministry in developing a 
strategic plan. Other municipalities could also find 
them helpful as a guide to best practices. 
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RECOMMENDATION 1 


To better define and fulfill the province’s roles 
for ensuring sustainable, well-maintained social 
housing, the Ministry of Municipal Affairs and 
Housing should: 

@ establish a comprehensive strategic plan that 
includes measurable goals and performance 
outcomes; 

@ work with municipalities to ensure a co- 
ordinated and integrated housing strategy 
within the province, and gather the informa- 
tion necessary to monitor progress on the 
strategy and on the goals and outcomes 
established; and 

@ consider requiring all Consolidated Muni- 
cipal Service Managers to develop local 
strategic plans, and encourage the sharing of 
best practices in developing such plans. 





The Ministry agrees and has already com- 


pleted over 13 public consultations with key 
stakeholders across the province to initiate the 
development of a Long-term Affordable Housing 
Strategy. As part of the exercise in developing 
the Strategy, the Ministry has been working 
with its municipal partners to develop social- 
housing performance measures. The Ministry 
expects to release the Strategy in 2010. The 
Ministry will collect the information necessary 
to monitor progress on the Strategy and on the 
goals and outcomes established. As part of the 
Strategy, the Ministry will consider requiring 
municipal service managers to develop local 
strategic plans. 

The Ministry currently requires municipal- 
ities to report annually financial and statistical 
information on various municipal services they 
provide. In addition, social housing perform- 
ance measures will be established and all 
municipalities will be required to report on the 
measures annually. 
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SUFFICIENT AND WELL-MAINTAINED 
SOCIAL HOUSING 


Effective provincial oversight is needed so that the 
Ministry can make informed funding decisions and 
take appropriate, timely action on systemic issues 
affecting the provision of social housing. Further, 
good management information is needed to sup- 
port the Ministry’s oversight activities. As indicated 
in the previous section, the level of provincial 
oversight had been minimal since devolution. Con- 
sequently, there was little management information 
available at the Ministry. We identified a number 
of significant issues that had not been adequately 
addressed since devolution. 

The following sections contain a discussion of 
some of these issues. 


Waiting Lists for Social Housing 


Much of the information available to the Ministry 
came from the annual Returns submitted by Service 
Managers. In addition to financial information, 
Service Managers are asked to confirm that they 
complied with the provincial requirement for main- 
taining the prescribed number of housing units and 
to provide supplemental information. 

One piece of supplemental information provided 
by Service Managers related to prospective ten- 
ants on waiting lists, which, according to the latest 
Returns, stood at about 137,000 province-wide as of 
December 2008. 

However, there was only limited additional 
information at the Ministry about the waiting lists. 
During our audit, we gathered additional informa- 
tion on the breakdown and length of the waiting 
list, as shown in Figure 1. 

Overall, the size of the province-wide waiting 
list has remained fairly stable over the past five 
years. However, the list could include households 
that have given up because of the lengthy wait 
times and others that could have been placed on 
more than one list. Many urban centres have wait 
times of five or more years for new applicants—and 


one reported wait times of up to 21 years for all 
applicants except seniors. 

Conversely, we noted that some Service Man- 
agers reported vacancy rates ranging from 0% to 
more than 5%, with a few smaller jurisdictions 
as high as 12%. Several also had a high vacancy 
rate despite a lengthy waiting list. Although some 
vacancies are temporary and unavoidable, a per- 
sistently high vacancy rate could be attributable 
to such reasons as lack of demand, a mismatch 
between size of units and demand, or, in extreme 
cases, units that were unusable because of poor 
safety and sanitary conditions. The Ministry did not 
monitor or assess the wait times and vacancy rates 
being reported by the individual Service Managers. 

Some Service Managers and jurisdictions have 
developed extensive information processes to 
assess the demand and supply for social housing. 
For example, information on the number and 
types of housing units, condition of housing stock, 
vacancies, and availability of special-needs housing 
would be useful to the Ministry in assessing social 
housing supply from a province-wide perspective. 
Data on population, by segment, number and com- 
position of households, and income factors would 
also be useful in analyzing housing demand. Such 
information is critical for improving future housing- 
program design to ensure that limited funds are 
directed to the areas of greatest housing needs. 


Co-ordination of Social Housing with Other 
Support Services 


The order in which people on a waiting list get 
housing is chronological—first-come, first-served. 
The one exception to this rule under the Social 
Housing Reform Act is for special-priority appli- 
cants, who are primarily victims of abuse. Service 
Managers at some of the larger municipalities 
indicated that as units became available, between 
25% and 40% of them were currently being occu- 
pied by these special-priority tenants, who require 
support services. Due to the urgent nature of their 
situations, the applicants on this list justifiably 
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Figure 1: Selected Wait-list Information, at December 2008 


Source of data: Selected Consolidated Municipal Service Managers 


Service Total # on 
Manager Waiting List 





A 66,600 4.5-6 45 
B 13,328 7-21 7 
C 9,691 2.5-4.5 20 
D 5,833 7-10 7 
E 1,564 0.5-8 0.5 
F 1,447 4-13 6 


receive priority for housing placement. Neverthe- 
less, Service Managers indicated that many of these 
households also require complex social-support 
services and additional security arrangements, the 
provision of which most housing providers do not 
have the capacity to ensure. 

In addition to special-priority housing, there 
were three provincial ministries that administer 
more than 20 other housing and related programs, 
including emergency shelters, housing for people 
with special health needs, and accommodations for 
the homeless. Service Managers indicated it had 
been difficult to co-ordinate their work and ensure 
special-needs tenants received the appropriate sup- 
port services. Their view was reflected in a recent 
Provincial-Municipal Fiscal and Service-Delivery 
Review, which reported that the current system is 
a fragmented and inefficient approach to meeting 
client needs. 


RECOMMENDATION 2 


To help provide sufficient social housing effi- 
ciently and make the most of available funding, 
the Ministry of Municipal Affairs and Housing 
should work with Consolidated Municipal Ser- 
vice Managers to: 

e establish more comprehensive reporting of 
information on social-housing portfolios 
and wait times so this can be taken into con- 
sideration in addressing the housing needs 


of individual municipalities; 


Expected Wait Time for New Applicants (Years) 





4.5 6 
21 va! 
4.5 d 
10 9 

8 2 
13 4 


@ identify ways to better and more equitably 
address the issue of lengthy wait times in 
many municipalities; and 

@ better co-ordinate housing and other support 
services with other provincial and municipal 
stakeholders. 


The Ministry agrees and will consider the Aud- 
itor General’s recommendation in the develop- 
ment of its Long-term Affordable Housing 
Strategy. While the Ministry currently collects 
information required to meet the terms of the 
Social Housing Reform Act, 2000, and its regula- 
tions, it will work with municipalities to identify 
and address other areas where additional and 
consistent information can benefit. 

Support services are provided at the local 
level. In keeping with the direction given under 
the recent Provincial—Municipal Fiscal and 
Service-Delivery Review, the Ministry will work 
together with municipalities to develop a con- 
solidated housing service to better co-ordinate 
housing and other support services with other 
provincial/municipal stakeholders. Provincial- 
municipal discussions are under way on this 
topic. 
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Social Housing Portfolio 


Like any other properties, buildings in the social 
housing portfolio require regular maintenance 
and periodic replacement of major capital items, 
such as roofs, underground garages, elevators, and 
mechanical systems to maintain and prolong their 
service life because it costs far more to build new 
units than to properly maintain existing ones. 

Prior to downloading, all public-housing units 
were owned by the Ontario Housing Corporation 
and capital requirements were funded through the 
Corporation’s annual budgeting process. In the 
case of privately owned non-profit and co-operative 
housing, each housing provider was required 
under its operating agreement with the province 
to contribute a portion of the operating subsidy to 
a separate capital-reserve fund to address capital 
repairs and replacements. 

After the decision was made to transfer social 
housing to the municipalities in the late 1990s, 
the Ministry commissioned studies to determine 
the condition of the housing stock and the related 
capital-funding requirements for the public, non- 
profit, and co-operative portfolios. These studies 
concluded that the social housing stock was in 
good condition overall. The required annual cap- 
ital funding for the publicly owned portfolio was 
determined to be in the $100-million range. About 
$52 million of this would be funded through the 
federal annual contribution and the remainder by 
Service Managers. 

With respect to the non-profit and co-operative 
housing providers, their previous operating agree- 
ments with the province were terminated at devolu- 
tion. The Social Housing Reform Act requires these 
housing providers to continue contributing to a 
capital-reserve fund. Except for a few large muni- 
cipalities, their contributions were being managed 
by the Social Housing Service Corporation, cre- 
ated under the Social Housing Reform Act in 2002. 
Examples of the services that the Corporation pro- 
vides include the pooling of capital reserves, group 
insurance, and bulk purchasing. 


Condition of Social Housing Stock 
Notwithstanding the reasonably good condition of 
the social housing stock at the time of devolution a 
decade ago, social housing stock has deteriorated 
since that time, particularly those properties within 
the public-housing portfolio. This has been a signifi- 
cant and growing concern to municipalities because 
the average property in the public-housing portfolio 
is close to 40 years old and capital maintenance 
costs are rising more quickly. 

While this concern has been identified by vari- 
ous stakeholders, there was a lack of up-to-date and 
reliable information on the province-wide condi- 
tion of housing stock. Currently, such information 
would only be available if housing providers carried 
out their own building-condition assessments, 
which provide estimates of the cost and time frames 
for repairing or replacing various building ele- 
ments. The results could then be used, for instance, 
to create a capital-reserve fund to cover replace- 
ment costs. For example, the Toronto Community 
Housing Corporation, which owns 60,000 social 
housing units accounting for two-thirds of Toronto’s 
social housing stock, carried out such an assess- 
ment in 2006 and estimated that $300 million was 
needed for immediate capital repairs. However, the 
Ministry did not have any information on how this 
estimate was determined or the magnitude of this 
issue on a province-wide basis. 

The average age of the non-profit housing port- 
folio was lower than that of public housing. Many 
non-profit projects were built in the late 1980s to 
early 1990s, resulting in an average age of about 18 
years. In addition, the non-profit housing providers’ 
agreements with Service Managers were structured 
to require reserve funds for future capital repairs. 
As of 2007, the Social Housing Services Corporation 
invested and administered approximately $400 mil- 
lion in capital reserves on behalf of non-profit 
housing providers. Nevertheless, the Corporation 
indicated that some non-profit providers were in 
crisis and most of the properties were just reaching 
the stage in their life cycle where major repairs 
would be necessary. However, on an overall basis, 


the non-profit housing portfolio was in better 
condition and should be better able to address its 
capital requirements than should public housing. 
Service Managers indicated that, as the housing 
stock continues to age, access to suitable financing 
to help pay for maintenance and repairs was 
emerging as a major issue. One potential source of 
financing, for example, would be to re-mortgage 
properties. However, the province had generally not 
agreed to housing providers’ proposals to refinance, 
citing the province’s potential liability under the 
Social Housing Agreement that requires Ontario to 
compensate CMHC for any costs arising from the 
default of housing providers. In the opinion of some 
Service Managers it is they, and not the province, 
who are responsible for the costs of such defaults. 
The Ministry should re-assess the implications 
of the Service Managers’ refinancing proposal to 
determine if it may be a viable solution. 


Asset Management 
Good asset-management practices, including 
regular preventive maintenance, are essential to 
prolong the life of housing assets and avoid costly 
repairs in future. In addition to lacking information 
on the condition of the social housing stock, the 
Ministry had no information about whether Service 
Managers have established good asset-management 
practices and whether housing providers were 
following them. The poor condition of some prop- 
erties could be due to delays in carrying out the 
regular maintenance required to prolong the life 
of the assets. Due to increasing operating costs, we 
understand that some housing providers may be 
redirecting funds away from regular maintenance 
to fund more urgent day-to-day operations. 

Energy-efficiency upgrades could also help 

free up some funds for maintenance. Older public- 
housing projects, for example, were built using less 
modern technology. Upgrading these buildings to 
contemporary energy-efficiency standards would 
generate operating-cost savings that could be used 
for other required capital-maintenance projects. 
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The Ministry has recently initiated the develop- 
ment of an asset-management strategy with the 
Social Housing Services Corporation to help hous- 
ing providers improve their practices in this area. 
Although it was in the early stages of development, 
this is a much-needed initiative. 


RECOMMENDATION 3 


To ensure that the housing stock is safe and of 
acceptable quality and that it will achieve its 
expected service life, the Ministry of Municipal 
Affairs and Housing should work with Consoli- 
dated Municipal Service Managers to: 
® carry out periodic building-condition assess- 
ments and ensure that such information is 
summarized on a province-wide basis; and 
@ develop an effective funding and financing 
strategy for raising the capital investment 
required to reduce the capital maintenance 
backlog and sustain proper maintenance of 
housing stock, including consideration of 
requirements that a capital reserve be estab- 
lished for public-housing stock. 

The Ministry should also continue to work 
with the Social Housing Services Corporation to 
assess the cost/benefit of implementing modern 
energy-efficient measures, and facilitate adop- 
tion of such measures by housing providers. 


In 2008, the Ministry helped fund the estab- 
lishment of an Asset Management Centre for 
Excellence by the Social Housing Service Cor- 
poration to provide the support and expertise 
that social-housing providers can draw upon 

in maintaining their buildings. The Centre 
promotes and recommends best practices for 
building condition audits and other capital plan- 
ning activities. Service Managers may choose to 
use some of the funding under the new Social 
Housing Renovation and Retrofit program, 
established in 2009 to upgrade social-housing 
projects, for building condition audits. 
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With respect to raising the capital invest- 
ment required to reduce the capital mainten- 
ance backlog, under the government’s Poverty 
Reduction Strategy announced in 2008, most 
social-housing providers are now able to 
apply to Infrastructure Ontario for low-cost 
capital loans. In addition, an Asset Leveraging 
Working Group co-chaired by the Ministry 
with the Social Housing Services Corporation 
is developing proposals for refinancing and 
renewing the social housing portfolio. 

With respect to energy-efficient measures, 
there are minimum energy-efficiency require- 
ments under the new $704-million Social 
Housing Renovation and Retrofit Program. 
Specifically, $70 million of this new federal— 
provincial funding is targeted for renewable 
energy initiatives. The Ministry will continue to 
consider other options for energy efficiency in 
the sector. 


FEDERAL FUNDING OF SOCIAL HOUSING 


As part of the Social Housing Agreement, the 
federal government began providing annual block 
funding to the province in 1999. The funding 

has several components, including capital costs, 
ongoing operating costs, and debt-servicing costs. 


The first annual payment amounted to $525 million 
and total subsequent payments decreased each year 


as the federal government’s operating agreements 
with housing providers began to expire. Starting in 
2015, a large number of agreements will begin to 
expire, which means Service Managers will have to 
maintain the same number of units with declining 
federal funding, as shown in Figure 2. 

The province’s Social Housing Reform Act 
requires Service Managers to maintain a prescribed 
minimum number of rent-geared-to-income units 
despite the eventual end of federal funding. No 
agreement has been reached with the federal gov- 
ernment for continued funding and all agreements 


will have expired by 2032. This issue is critical for 
some municipalities, who derive the bulk of their 
social housing budgets from federal funding. 

Under these circumstances, some housing pro- 
jects would no longer be financially viable because 
rental revenues are insufficient to cover operating 
costs, even when the property is mortgage-free. 
Properties housing a high proportion of house- 
holds that pay low rent would be affected the 
most because they depend largely on government 
subsidies to meet operating costs. The expiry of 
these funding agreements without commitments 
to establish new ones could force non-profit and 
co-operative housing providers to stop providing a 
large number of social housing units. The majority 
of Service Managers who responded to our survey 
noted that they have yet to find a solution to deal 
effectively with this issue. The Ministry indicated 
the province has had ongoing discussions with the 
federal government regarding the pending decline 
in federal funding. However, there has not been any 
commitment from the federal government to renew 
the funding and the Ministry had not developed a 
contingency plan to address this situation. 

With respect to the provincial requirement for 
maintaining the prescribed minimum number of 
housing units, Service Managers indicated that the 
numbers were imposed by the province at the time 
responsibility for social housing was handed over to 
municipalities to protect the level of existing hous- 
ing stock. Some Service Managers said that both the 
number and composition of housing units has never 
reflected the demographics and housing demand of 
municipalities, particularly as it has been nine years 
since the province downloaded this responsibility. 

Service Managers also noted a difference 
between what the federal government transferred 
to the province under the Agreement and what the 
Service Managers received from the province, even 
though the Agreement stipulates federal funding 
must be used for eligible housing programs. 

Our analysis of data supplied by the Ministry 
for the years 2000 to 2008 indicated that the 
federal government forwarded about $4.8 billion 


Social Housing 


Figure 2: Expiry of Funding Agreements with Federal Government for Social Housing, 2000/01-2033/34 (Units) 


Source of data: Ministry of Municipal Affairs and Housing 


250,000 





200,000 


150,000 









100,000 e=@= units funded 


wae Minimum service level/units requirement 





50,000 


0 ne ee 
Se eo ee ee at) eS So SH st 10 (Orr OVO Petia? ee I eS 80d) os ea en = 
Dene ope se ue C GENE CN ACs CN GN CNN CN EN ON) CCS A ENON ce Didy ad Cop ENG leg andy CRONE pe 
to Ontario for social housing during that period. The Ministry explained that Service Managers’ 
Of that, approximately $3.8 billion was allocated social housing expenditures and not just those of 
to Service Managers and their predecessors in the the province were counted towards fulfilling the 
period before devolution took full effect in 2001 for | Agreement’s term that federal funds be spent only 
eligible programs under the Agreement. Another on eligible social housing programs. As long as 
$414 million was allocated to other provincial the combined annual social housing expenditures 
and municipal social housing programs under the of the Ministry and Service Managers exceeded 
Agreement. For the remaining federal transfer of the total federal transfer, the Ministry considered 
about $620 million, ministry figures show: itself to be in compliance with the Agreement. The 
e Approximately $290 million was allocated to Ministry indicated that its legal counsel had con- 
another provincial social housing program; firmed that there were no legal concerns about this 
and arrangement. 
e The province withheld $330 million, of which Nevertheless, we noted that although the 
it designated $198 million—$22 million a $3.8 billion transfer to Service Managers was pub- 
year for nine years—as what it called a “prov- lished in the Ontario Gazette, the Ministry had not 
incial constraint.” The Ministry was unable to disclosed the use of the remaining federal funds. 
provide support that this and the remaining Therefore, Service Managers generally were not 
$132 million were spent on housing. aware of the portion of federal funds spent on other 


housing programs. Consequently, they questioned 
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the difference between what the federal govern- 
ment transferred to the province and what they 
actually received. 





| RECOMMENDATION 4 


To mitigate the possible impact of continuing 
decreases in federal funding on the supply of 
social housing, the Ministry of Municipal Affairs 
and Housing should: 

_@ develop a plan for options, should nego- 
tiations with the federal government for 
continued funding for the social-housing 
portfolio be unsuccessful; 

e work with Consolidated Municipal Service 
Managers on alternatives to the current 
system of maintaining the required number 
of housing units with an aim to better match 
the supply of social housing to the demand 
in each municipality; 

® review its current methodology to ensure 
funding allocations are fair and federal funds 
are spent on eligible housing programs; and 

e provide a full and public accounting of how 
all federal funding provided for social hous- 
ing was spent. 





The Ministry, in co-operation with other prov- 


inces and territories, continues to raise this issue 
with the federal government. 

The consensus recommendations of the 
Provincial—Municipal Fiscal and Service-Delivery 
Review have supported the local responsibil- 
ities for social housing. Once expenditures 
identified to be transferred to the province have 
been implemented, municipalities will have 
$1.2 billion a year in net benefits so that they 
can respond to social housing and other local 
priorities. 

The Ministry will work with service man- 
agers to support them in understanding the 
levels of discretion they currently have to 
change the composition of social housing units 


with their existing funding, and still meet their 
service standards requirement under the Social 
Housing Reform Act, 2000. 

The Ministry will review its current method- 
ology for federal funding allocations, including 
principles of fairness. We are consulting on 
a long-term affordable housing strategy that 
could include future funding relationships. 

The Ministry currently provides accounting 
of its revenues and expenditures according to 
the Printed Estimates and Public Accounts pro- 
cess, and reports annually to the CMHC on how 
the federal funding received by the province 
under the Social Housing Agreement is spent. 
The Ministry will consider how best to provide 
further information on how federal funding 
received under the Agreement is spent. 


RECENT FUNDING INITIATIVES 


In partnership with the federal government, the 
province recently began to provide Service Man- 
agers with funding for new housing programs. We 
noted that the Ministry had sufficient controls in 
place to ensure Service Managers complied with 
the requirements of these new programs, but we 
found there was little information or front-end 
analysis in place to assess these new funding 
initiatives and ensure they would have the desired 
impact on social housing. There were, for example, 
no business cases that detailed all the costs and 
benefits of the new programs. In addition, there 
were virtually no accountability or reporting 
requirements for measuring the impact of funding 
provided. The following are some examples from 
our review of some of the new programs. 


Delivering Opportunities for Ontario 
Renters [DOOR] 


DOOR was a one-time, $127-million capital grant 
program to Service Managers in the 2006/07 fiscal 


year to support the delivery of housing for low- 
income households. Grants were made without 
condition or reporting requirements, leaving the 
Ministry unable to determine whether funds were 
being spent fulfilling the most critical social hous- 
ing needs. 


Social Housing Capital Repairs 


In its 2008 budget, Ontario announced investments 
in municipal infrastructure that included $100 mil- 
lion to address urgent capital-repair needs of the 
social housing stock. However, as the Ministry had 
no information on the condition of the social hous- 
ing portfolio, allocations were made in the 2007/08 
fiscal year based on each Service Manager’s propor- 
tion of units in the social housing portfolio rather 
than on actual need for capital repairs as deter- 
mined by ongoing condition assessment and sound 
asset-management practices. 


Affordable Housing Programs 


The CMHC defines households as being in core 
housing need if their dwelling falls below certain 
standards of adequacy, suitability, or affordability, 
which is defined as rent not exceeding 30% of 
gross total household income. In 2002, the Ontario 
government introduced the Canada—Ontario 
Affordable Housing Program (Program) in partner- 
ship with the federal and municipal governments. 
The Program, intended to create more than 18,000 
units of affordable housing for households on social 
housing lists, or those eligible to be on such lists, 
included several components and had a total fund- 
ing commitment of $624 million as of March 31, 
2009. The federal government contributed 

$348 million of the total while the province con- 
tributed the remaining $276 million. New afford- 
able housing would be created primarily through 
construction of new rental units, home ownership, 
and through rent supplements to landlords. In 
Northern Ontario, funds would also be available to 
assist with home repairs. Our audit focused on two 


Social Housing . 289 


components of the Program, as described in the fol- 
lowing sections. 


Rental and Supportive Program 
The Rental and Supportive Program (Program) was 
intended to promote construction of new rental 
units for low- to moderate-income households 
by providing up to $70,000 per unit. Currently, a 
federal contribution of up to $26,600 per unit was 
to cover capital costs primarily, while the provincial 
contribution of up to $43,400 per unit would pay 
mortgage and operational costs so as to keep rents 
affordable for 20 years. As of March 2009, com- 
mitments to the Program were about 10,000 units 
costing $498 million. 

Under the Program, housing providers would 
for 20 years charge affordable rent, defined as 80% 
of the average market rent as determined each year 
by the CMHC. Service Managers were required 
to establish maximum income levels under which 
households qualify for these units, along with a 
process for verification of tenant incomes. 

According to Program guidelines, rental- 
housing-unit allocations were determined using 
a formula that took into account core housing 
needs and population growth. In addition, Service 
Managers were required to submit for the Ministry’s 
approval a Housing Delivery Plan that addressed 
areas such as the housing groups to be targeted, 
eligibility criteria, and the strategy for take-up and 
delivery of the units. This was a good basis for mak- 
ing such allocation decisions and we found that the 
proposed Housing Delivery Plans contained com- 
prehensive information on the municipalities’ hous- 
ing projects and their strategy. However, other than 
a checklist, there was no evidence that the Ministry 
had evaluated the Housing Delivery Plans and used 
the excellent information contained within to make 
their allocation decisions. 

In addition, Service Managers had to report on 
their projects annually—but the only information 
required of them was the number of units occupied, 
and assurances that they were charging 80% of 
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average market rent for the units. They were not 
required to report on their success in meeting 
Program objectives or their Housing Delivery Plan. 
As well, the Ministry had not ensured that Service 
Managers submitted reports on a timely basis as 
required. 

We noted that the Program objective of achiev- 
ing 80% of the CMHC’s average market rent for new 
units would be met, and that the overall supply of 
affordable housing did increase. However, many 
people on the waiting list could not afford the 
rent. The CMHC definition of affordable housing 
stipulates that households should not have to pay 
more than 30% of their pre-tax income on rent. We 
determined that more than half of the units in this 
Program would still be unaffordable for households 
on waiting lists, or eligible to be on the lists. For 
example, the average income of households on the 
waiting list in 2008 was $15,000, putting their max- 
imum affordable rent at $375 a month. Our analysis 
showed that average rent for the new units was 
$715 per month, meaning that households would 
need incomes of at least $29,000 annually (on aver- 
age) to consider these units affordable. The program 
may therefore assist moderate-income households 
but will do little for low-income households. 


Housing Allowance/Rent Supplement Program 
The Housing Allowance/Rent Supplement Program 
(Program) was established in 2005 to help bridge 
the gap between the rent that a household can 
afford and the actual market rent. This program 
was intended for lower-income households on, or 
eligible to be on, the waiting list. All funding for the 
program was provided by the federal government, 
which originally committed $80 million for 5,000 
units to be paid out in five-year agreements span- 
ning the years 2005 to 2013. However, by March 
2008, the deadline for committing to new agree- 
ments, the take-up had been just $57 million and 
3,721 units. 

Service Managers attributed the relatively low 
participation level to concerns with the Program’s 


design. For example, they cited the fact that under 
the Program, the agreements could last just five 
years and apply only to vacant units. That left 
prospective tenants facing the expense of having to 
move again in five years when the subsidy ran out. 
Service Managers said they would have preferred 
the supplements be provided to households for 
existing units rather than require them to move to 
vacant units. 


FUTURE FUNDING INITIATIVES 


In addition to the above recent funding initiatives, 
in March 2009, as part of the federal—provincial 
economic stimulus package, both levels of govern- 
ment announced an increase in their investment 

in social and affordable housing of more than 

$1.2 billion over the next two years, to be funded 
equally by the two levels of government. As indi- 
cated earlier, the Ministry’s efforts had focused on 
ensuring Service Managers complied with program 
requirements. It had not established and dedicated 
staff resources toward monitoring the success of 
its funding programs in achieving their desired 
impact. Improvements to the Ministry’s system of 
monitoring funding programs are needed to ensure 
these funds are spent cost-effectively. 


RECOMMENDATION 5 





To ensure that funding provided achieves the 
desired social housing impact, the Ministry of 
Municipal Affairs and Housing should require 
that: 
@ each new funding program is supported by a 
detailed business case; and 
@ adequate accountability mechanisms for 
reporting on the results achieved by Service 
Managers for the funds provided be put in 
place for all funding programs. 
In addition, the Ministry should make any 
necessary changes to ensure it has the resources 
and organizational capacity to properly monitor 
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the effectiveness of funding it provides to Ser- reporting on results by municipalities. Where 
vice Managers. adequate accountability mechanisms are lack- 


ing, the Ministry will work with them to ensure 
BEINISTRYRESPONSERIY ey aren pace 


The Ministry recognizes the need to review 
its resources and organizational capacity in light 
of the recent funding initiatives and directions 
provided under the Provincial—Municipal Fis- 
cal and Service-Delivery Review. The Ministry 
will assess its current resource requirements to 
undertake its obligations and monitor the effect- 
iveness of funding provided to Service Managers. 


The Ministry is required to submit a business 
plan to Treasury Board as part of its requests 
for program funding. It will review its current 
practice in developing business cases, to identify 
and implement any improvements as may be 
required to ensure a detailed business case is 
developed for each funding program. 

The Ministry will review the existing 
accountability mechanisms established for 
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Section 


Services 


Background : 


Ontario’s teletriage services provide callers with 
free, confidential telephone access to a registered 
nurse for health-care advice and information. The 
services comprise Telehealth Ontario, which was 
launched province-wide in 2001, and the Telephone 
Health Advisory Service (THAS), which was 
launched in 2003. 

The objectives of Telehealth Ontario and THAS 
include: 

@ facilitate the use of the most appropriate 

health services; 

@ improve access to appropriate health informa- 

tion and advice; 

@ increase health education and improve 

decision-making by consumers; and 

@ improve satisfaction with access to quality 

health information. 

Telehealth Ontario is available to all callers 24 
hours a day, seven days a week. THAS is available 
Monday to Friday, 5 p.m. to 9 a.m., as well as all 
day on weekends and holidays, to the 8.4 million 
patients enrolled, as of March 2009, with phys- 
icians participating in various primary health-care 
arrangements, such as family health teams. With 
either program, the nurses use their clinical judg- 
ment in conjunction with medical decision support 


Cha CMH Ministry of Health and Long-Term Care 
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software to assist callers (for example, by providing 
details on self-care, or by advising them to see their 
physician or go to the emergency department at 
their local hospital). Nurses can also provide callers 
with general health information or forward calls to 
poison control. As well, Telehealth Ontario callers 
can be linked to pharmacists at the Medication 
Information Service for drug-related inquiries. For 
THAS callers, nurses can access the on-call phys- 
ician from the caller’s physician’s practice, and, if 
needed, the physician may speak directly with the 
caller. 

The Ministry of Health and Long-Term Care 
(Ministry) contracts with a private service provider 
to deliver both Telehealth Ontario and THAS. The 
service provider employs almost 300 registered 
nurses at its five call centres located throughout 
Ontario. During the 2008/09 fiscal year, 905,000 
calls were responded to by nurses, and payments to 
the service provider totalled $35.1 million. 


Audit Objective and Scope 





Our audit objective was to assess whether teletri- 
age services were providing confidential access to 
timely advice in an economical manner that met the 
health-care needs of Ontarians. Our audit focused 
on Telehealth Ontario and THAS, and excluded 


other telemedicine services, such as physicians 
using telecommunications to provide health-care 
services. 

Our audit work was conducted primarily at two 
of the service provider’s call centres, because all 
calls are handled in one virtual queue, with the first 
available nurse at any of the call centres answering 
the call. In conducting our audit, we reviewed 
relevant files and administrative policies and pro- 
cedures, and met with appropriate staff from the 
service provider and the Ministry. In addition, using 
2008 data from the service provider, we reviewed 
call volume and caller wait times. We also reviewed 
relevant research and obtained information from 
comparable teletriage services programs in other 
Canadian jurisdictions. As well, with the assistance 
of an independent survey firm, we surveyed 1,100 
people across Ontario to obtain information on 
their awareness of and satisfaction with Telehealth 
Ontario’s services. 

We did not rely on the Ministry’s internal audit 
service team to reduce the extent of our audit work 
because it had not recently conducted any audit 
work on teletriage services. The service provider 
did not have an internal audit function. 





The Ministry appropriately contracted for the 


delivery of teletriage services based on a competi- 
tive process. The contract included a number of 

key performance requirements, focusing primarily 
on the timeliness of access to services, which the 
service provider reported that it met in the 2008/09 
fiscal year. As well, our independent survey indi- 
cated that those who used Telehealth Ontario were 
generally satisfied. However, only a small portion of 
Ontario’s population uses the services. In addition, 
based on our analysis of information at the service 
provider, as well as practices in other jurisdictions, 
there are improvements that could be made to 
enhance the services for Ontarians—for example, 


Teletriage Health Services L293 


adopting an easily remembered telephone number 
such as “811”; ensuring that newly hired nurses 
have the required clinical experience; conducting 
independent reviews of the quality of the advice 
provided by nurses; and establishing performance 
requirements for the quality of that advice. Some 
of our more significant observations included the 
following: 

e Not only has the number of calls to teletriage 
services been declining over the last few years, 
but the number of calls as a proportion of the 
population is significantly less in Ontario than 
is the case in Alberta and Quebec. Ontario 
had 905,000 calls to teletriage services in the 
2008/09 fiscal year, out of Ontario’s popula- 
tion of 12.2 million, while Quebec’s Info-Santé 
received 2 million calls out of a population of 
7.4 million and Health Link Alberta received 
1 million calls out of a population of 3.3 mil- 
lion. One reason may be that the Ministry did 
minimal advertising of the services during 
2007 and 2008. We also noted that the num- 
ber of calls concerning seniors as a proportion 
of Ontario’s senior population was low—only 
4% (72,000 calls out of a population of 
1.6 million). 

e Although over 60% of Ontario’s population 
was enrolled with physicians participating in 
various primary health care arrangements, 
and therefore eligible to use the Telephone 
Health Advisory Service (THAS), only 1% 
of eligible individuals used the service in 
2008/09. 

e British Columbia and Quebec use the eas- 
ily remembered “811” phone number for 
their teletriage services, and certain other 
provinces are planning to adopt this phone 
number. Quebec reported a 15% increase in 
call volume following its implementation. At 
the time of our audit, Ontario had no plans to 
adopt the “811” phone number. 

e The service provider does not track how long 
callers wait in the live queue before speak- 
ing to a nurse but indicated that about 25% 


Chapter 3 * VFM Section 3.13 





204 2009 Annual Report of the Office of the Auditor General of Ontario 





0 | 
oe 
oe) 
i= 
S 
= 
[+] 
® 
” 
= 
Le | 
— 
e 
o 
fe 
®o 
oS 
= 
© 
= 
oS 


of callers waiting in the live queue hung up 
before their call was answered by a nurse. Of 
the callers in the live queue who continued to 
wait, we calculated that 85% spoke to a nurse 
within 23 minutes. Eighty-five percent of 
callers who left a call-back number spoke to a 
nurse within 34 minutes. 

Physicians who were on call to THAS had 

to be paged more than once in over 70% of 
calls requiring a page during 2008, and 9% of 
pages were never returned. Further, 10% of 
physician practices did not return at least one- 
third of the pages they received. Generally, the 
on-call physician is paged if the nurse would 
advise the caller to see their doctor within four 
hours or go to the emergency department. 
Although advice to callers deviated from the 
clinical guidelines and protocols only 5% of 
the time in the 2008/09 fiscal year, almost 
30% of the deviations did not indicate the 
reason for not following the guidelines. 
Although the service provider had indicated in 
its proposal submitted to the Ministry in 2007 
that its nurses would have at least three years 
of any type of nursing experience, its policies 
require nurses to have one to three years of 
clinical experience. We noted from our sample 
of nurses hired in 2008 that 23% had less 
than one year of acute-care experience, and 
20% had less than three years of total nursing 
experience. 

Because callers are not asked to provide their 
Ontario health card number to the service 
provider, it is difficult to confirm whether call- 
ers actually followed the nurses’ advice. If the 
health card number was requested, it would 
be possible in many instances to check Ontario 
Health Insurance Plan records to determine 
whether the caller followed the advice given, 
and therefore whether the teletriage services 
were influencing callers to use the most appro- 
priate health service, as intended. 

Although there was no independent review of 
the quality of the advice provided to callers, 


the service provider estimated that the advice 
provided to about 95% of callers annually 
was appropriate. We noted that another Can- 
adian jurisdiction uses mystery callers, with 
predetermined questions, to independently 
evaluate the quality of advice provided. 
Because calls are generally not taped, the 
service provider’s quality assurance reviewers 
audited calls only as they were taking place. 
We found that 84% of the call audits sampled 
were completed during off-peak periods, 
rather than during peak periods (when nurses 
are under pressure to respond to waiting call- 
ers within established time frames). Most of 
the other provinces we spoke with indicated 
that they tape all calls, so that they can be 
reviewed at a later date to assess the appro- 
priateness of the medical advice given, among 
other things. 

Our independent satisfaction survey indicated 
that 82% of callers were satisfied or very satis- 
fied, which was slightly lower than the service 
provider’s reported 98% satisfaction, based on 
callers it sampled. However, neither the Min- 
istry nor the service provider had surveyed 
other stakeholders, such as family physicians 
and emergency departments, regarding their 
satisfaction. Other jurisdictions we spoke with 
indicated that health-care professionals’ sup- 
port and acceptance of teletriage services was 
critical to the success of their programs. 

In 2008/09, the Ministry paid the service pro- 
vider about $39 for each of the first 900,000 
registered calls to teletriage services, and 
about $27 per call after that. The three other 
provinces that shared cost information with 
us indicated that their teletriage services costs 
were about $20 per call. The Ministry had 

not conducted any work aimed at determin- 
ing the reason for the significant difference 
between Ontario’s costs and the costs in other 
provinces. 

Although the Ministry had established several 
good performance measures, which the 


service provider must meet in order to avoid 
financial penalties, the measures focused 
primarily on access to services. There were no 
performance standards relating to the quality 
of nurses’ advice or to the length of time call- 
ers wait in the live queue. 


The Ministry recognizes the importance of pro- 
viding confidential, timely access to teletriage 
services in an economical manner that meets 
the health needs of Ontarians, and welcomes 
the recommendations made by the Auditor 
General. As indicated in the responses to each 
recommendation, the Ministry will follow up 
on the Auditor General’s recommendations to 
enhance the existing safeguards and processes 
already in place. 


Detailed Audit Observations 





OVERVIEW 


All calls to the toll-free numbers for Telehealth 
Ontario and for THAS are answered by an auto- 
mated attendant that asks the caller to select a 
language (English or French). Callers are then 
placed into either the English or French centralized 
queue, which are answered directly by a nurse, if 
one is available, at any of the service provider’s 

five Ontario call centres. If all of the nurses are 
busy, the call is routed to a receptionist, known as a 
patient assistance representative. The receptionist 
obtains information from the caller (for example, 
name, phone number, and nature of problem), 
although callers can choose to remain anonymous. 
The service provider’s phone system is supposed 

to reject calls from outside Ontario and callers are 
not required to provide their Ontario health card 
number. The receptionist then gives the caller the 
option of remaining in the live queue to speak toa 
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nurse, or going into the call-back queue to have a 
nurse phone the caller back. If all the nurses and all 
the receptionists are busy, the caller reaches a voice 
message system that gives the caller the choice of 
remaining on the line or leaving contact informa- 
tion so a nurse can return the call. Receptionists 
pick up messages left in the voice-mail system and 
add them to the call-back queue. If a call is urgent, 
it may be placed in a priority call-back queue to be 
answered faster. Figure 1 illustrates the standard 
call-handling process. 

A summary of monthly statistics reported to the 
Ministry by the service provider indicated that, in 
2008/09, 34% of calls were answered directly by a 
nurse, 65% were answered by a receptionist, and 1% 
were answered by the voice message system. About 
60% of callers reaching the receptionist requested a 
call back, rather than waiting in the live queue. 

Once the caller reaches a nurse, the nurse uses 
decision support software, with medical algorithms 
that provide the nurse with guidelines and proto- 
cols for handling symptom-based calls. The nurses 
use the recommendations from this software in 
conjunction with their professional judgment to 
provide advice and information to the caller. In 
2008, a little more than half the callers were calling 
about themselves; the remaining callers were call- 
ing about someone else (usually a child or spouse). 

Where applicable, nurses obtain the caller’s 
consent to forward call information (for example, 
to a hospital emergency department, or in the case 
of THAS callers, to their physician). For THAS call- 
ers with more urgent needs, the nurse pages the 
physician who is on call from the caller’s physician’s 
practice. If needed, the physician may decide to 
speak directly with the caller. All calls are logged in 
the service provider’s system. 


ACCESS TO TELETRIAGE SERVICES 
Public Awareness 


In 2001, when Telehealth Ontario was launched, the 
Ministry conducted a promotional campaign that 
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included media advertisements and distributing campaign. With respect to THAS, the Ministry’s 
fridge magnets with the service’s phone number contracts with the various physician practices state 
to each household in the province. Subsequent that promoting THAS is a joint responsibility and 
specific promotional activities were conducted, with that physicians are to advise their patients about 
the most recent being in 2006 when the Ministry the service, for example, by prominently posting a 
distributed door hangers highlighting the avail- notice in their office. 

ability of translators for non-English-speaking and Between 2003 and 2006, the Ministry com- 
non-French-speaking residents. As well, Telehealth missioned a number of public surveys to assess 
Ontario is a component of the Ministry’s “Your the public’s awareness of Telehealth Ontario. The 
Health Care Options” campaign. Starting in 2009, surveys found that about two-thirds of respond- 
it was included in some advertisements for this ents were aware of the service. No further public 
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Figure 1: Teletriage Services Standard Call-handling Process 
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* Information to verify these numbers was not readily available from the service provider. 


surveys have been commissioned by the Ministry 
since 2006. 

As shown in Figure 2, in the 2008/09 fiscal year 
the teletriage services received more than 1.1 mil- 
lion calls, and nurses responded to 905,000 of 
those calls. Our analysis indicated that a maximum 
of 650,000 unique callers, or about 5% of Ontario’s 
12.2 million population, made those calls: that is, 
the total included about 250,000 calls made by 
individuals who had already called at least once 
before during that fiscal year. The service provider 
indicated that since the service’s inception about 
70% of callers have phoned more than once. 

We noted that similar programs in other prov- 
inces were called more frequently. For example, 
Quebec’s Info-Santé received 2 million calls in the 
2008 calendar year from a population of 7.4 mil- 
lion, and Health Link Alberta received 1 million 
calls in 2008 from a population of 3.3 million. 

We also noted that the number of Ontarians 
enrolled with physicians participating in various 
primary health-care arrangements, and therefore 
eligible to use THAS, tripled from 2.8 million in the 
2004/05 fiscal year to 8.4 million in the 2008/09 
fiscal year. As of March 31, 2009, THAS-eligible 
callers represented about 65% of Ontario’s popula- 
tion. However, as shown in Figure 3, virtually none 
of these eligible callers used THAS, but they may 
have called Telehealth Ontario instead. In 2008, 
THAS received calls concerning only about 1% of 
eligible individuals. 

As well, patients seem to use THAS more when 
their physicians support and promote the service. 
For example, we identified 18 primary health 
groups with more than 1,000 patients calling THAS, 
and almost 140 groups with fewer than 10 patients 
calling THAS. Practices sampled with more than 
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1,000 calls generally had an after-hours voice mes- 
Sage system that referred patients to THAS. Practi- 
ces with fewer than 10 calls generally did not refer 
patients on their after-hours voice message system. 

The service provider indicated that the decline 
in calls to the teletriage services shown in Figure 2 
was due to mild flu seasons and minimal promo- 
tion of the services. However, the service provider 
advised us that it would require additional nursing 
resources, which can be challenging to recruit, to 
answer the increased call volume that would result 
from additional promotion. 


Telephone Number 


To contact either Telehealth Ontario or THAS, call- 
ers dial each service’s respective toll-free telephone 
number. The number for Telehealth Ontario is 
typically listed in local phone books, but not neces- 
sarily at the front with other important phone num- 
bers. Eligible individuals receive the phone number 
for THAS from their physician’s office. 

In July 2005, the Canadian Radio-television 
and Telecommunications Commission set aside the 
phone number “811” for provinces to use for non- 
urgent health teletriage/telehealth services. At the 
time of our audit, British Columbia and Quebec had 
adopted “811” as the phone number for their tele- 
health services. Representatives from Quebec told 
us that their teletriage services experienced a 15% 
increase in call volume following the implementa- 
tion of the “811” phone number. As well, we were 
informed that certain other provinces are moving 
toward using “811.” 

At the time of our audit, the Ministry indicated 
that there were no plans for Ontario to adopt the 
“811” phone number for its teletriage services. 


Figure 2: Calls to Teletriage Services by Type, 2006/07-2008/09 


Source of data: Service provider 


Type of Call Description 
automated attendant completed message 


nurse provided advice or information 


incoming 
registered 





# of Calls 
1,305,000 1,207,000 1,145,000 
1,094,000 986,000 905,000 
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Figure 3: Eligible Callers and Actual Calls to the Telephone Health Advisory Service, 2004/05-2008/09 


Source of data: Service provider and Ministry of Health and Long-Term Care 














2004/05 PAYA) 2006/07 PAU AU) 2008/09 
# of registered calls to THAS 79,000 134,000 134,000 123,000 112,000 
# of eligible THAS callers 2,875,000 5,722,000 7,005,000 7,731,000 8,386,000 





registered calls to THAS as a % of eligible calls ta § 23 1.9 1.6 1.3 


Demographics 


Based on our analysis of caller data for 2008, 

37% of calls to the teletriage services concerned 
children, 51% concerned adults between 18 and 

64 years old, and 8% concerned seniors aged 65 
and older. The remaining 4% were anonymous. 
Although there are more than 1.6 million seniors in 


ing from the more populated areas of central and 
southwestern Ontario, as shown in Figure 4. 

The service provider indicated that the telephone 
system is designed to block calls from outside 
Ontario. However, using 2008 data extracted from 
the service provider’s information systems, we noted 
that almost 2,000 registered calls came from outside 
Ontario. These included calls from eight other prov- 
inces, as well as 19 U.S. states. The service provider 
told us that these calls are often either from persons 
using an Ontario cellphone from outside Ontario 
or visitors calling while vacationing in Ontario, 
who provide an out-of-province phone number. In 
addition, for about 10% of the calls, certain caller 
information (such as their address) was blank or 
incomplete. Because callers are not required to pro- 
vide such information, we were unable to determine 
whether these callers were from Ontario. 








Figure 4: Source of Calls to Teletriage Services by 
Region, 2008 


Prepared by the Office of the Auditor General of Ontario 
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RECOMMENDATION 1 


In order to provide more accessible teletriage 

health advice and information, the Ministry 

should: 

_@ consider the continued need for a separ- 

ate THAS service or options for increasing 
the level of awareness and acceptance of 
teletriage services, especially among indi- 
viduals eligible to use the Telephone Health 
Advisory Service (THAS) and among those 
demographic groups, such as seniors, that 
underutilize the services; and 

e explore the use of an easily remembered phone 
number, such as “11” (which is used or being 


planned for in several other large provinces), 
for both Telehealth Ontario and THAS. 


The service contracts for Telehealth Ontario and 
the Telephone Health Advisory Service were 
combined into one contract in 2008. This facili- 
tates combining the services, and the Ministry 
will explore this possibility. 

In August 2009, refrigerator magnets 
advertising Telehealth Ontario were distributed 
in over 200 community newspapers province- 
wide. As well, in September 2009, the Ministry 
planned to distribute to each household in the 
province a pamphlet regarding flu prevention, 
which will advertise Telehealth Ontario as a 
resource for callers seeking health informa- 
tion and advice. Further, Telehealth Ontario 
continues to be part of the Ministry’s Healthcare 
Options campaign, which includes television, 
online, and print advertisements. The longer- 
term impact on the public’s awareness of 
teletriage services and usage will be measured to 
determine any additional investments necessary. 

Past research by the Ministry has shown that . 
low-use groups, such as seniors and youths, 
were not interested in using Telehealth Ontario. 
However, the Ministry will further investigate 
the feasibility of additional initiatives targeted 
towards these groups, as necessary. Ontario’s 
possible use of 811 for teletriage services will 
also be explored. 


CALL MANAGEMENT 
Wait Times 


According to the contract with the Ministry, the ser- 
vice provider is required to meet certain perform- 
ance standards for responding to calls. To ensure 
that these standards are met, the service provider 
has individuals who monitor, among other things, 
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the time that individuals wait in the call-back 
queue. As shown in Figure 5, the service provider 
reported that it met all the performance standards 
for the 2008/09 fiscal year. 

If the receptionist determines a call to be a high 
priority, that call is responded to more quickly. In 
2008, about 14% of calls placed in the call-back 
queue were deemed high priority; in our sample, 
nurses phoned 90% of these callers within three 
minutes. 

Other calls are generally handled in the order 
received, with a nurse answering the caller who 
has been waiting the longest, whether that caller is 
in the live queue or the call-back queue. However, 
once the wait time in the call-back queue reaches 
about 15 minutes, certain nurses—for example, all 
nurses at one call centre—are told to answer only 
calls in that queue. If necessary, certain call centres 
are directed to take specific calls to minimize the 
number of calls that will not meet the perform- 
ance standards. For example, calls waiting 20 to 
29 minutes would be taken before those waiting 
over 30 minutes. Such situations are more likely to 
happen when call volumes are highest. As shown 
in Figure 6, call volumes are generally highest on 
statutory holidays and weekends, as well as in the 
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evenings. In addition, as shown in Figure 7, more 








Figure 5: Teletriage Performance Standards Effective 
April 1, 2008, and Results for 2008/09 


Source of data: Ministry of Health and Long-Term Care 


Result 


80% of all calls in each month should be met 
transferred from the automated attendant to a 

person (as opposed to a voice message system) 

within 20 seconds. 


85% of callers who leave contact information met 
for a call-back must receive the call-back from a 
teletriage nurse within 30 minutes 


98% of callers who leave contact information met 
for a call-back must receive the call-back from a 
teletriage nurse within 2 hours. 


Abandoned calls (that is, caller hangs up after met 
recorded greeting, but before speaking to a 

person or leaving a message) should not exceed 

6% of all incoming calls per month. 


Performance Standard 
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Figure 6: Average Number of Registered Calls by 
Hour of Day, 2008 


Prepared by the Office of the Auditor General of Ontario 
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calls are received during the winter months than at 
other times of the year. 

Using data from the service provider, we calcu- 
lated wait times for callers waiting in the live queue 
and the call-back queue from the time the call was 
received for the period October through December 
2008. We found: 

@ 85% of callers in the live queue spoke to a 
nurse in 23 minutes, and 90% spoke to a 
nurse in 36 minutes; and 

@ 85% of callers in the call-back queue spoke to 
a nurse within 34 minutes, and 90% of callers 
in the call-back queue spoke to a nurse within 
49 minutes. 

The time we calculated for the call-back queue 
is longer than that being reported by the service 
provider because we determined that the wait time 
for a call-back started when the incoming call was 
initially answered by the automated attendant. 

For its part, the service provider began measuring 
Wait time from the time the caller was put into 
the queue. This approximates the time the caller’s 
information was left, which, according to the 
service provider’s contract with the Ministry, is 
the beginning of the caller’s wait-time. As well, if 
a caller started to wait in the live queue and then 
decided to transfer to the call-back queue, the 
service provider would not capture the time spent 


Figure 7: Number of Calls by Month, 2008 


Source of data: Service provider 
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waiting in the live queue. The service provider did 
not track the time callers waited in the live queue. 


Paging On-call Physicians 


For individuals eligible to call THAS, the nurse 
may page the on-call physician from the caller’s 
physician’s practice. Physicians’ practices are paid 
up to $2,000 a month for being on-call, which is 
one of the services that physicians participating in 
various primary health-care arrangements agree to 
provide in their contract with the Ministry. The on- 
call physician is usually paged if the nurse, based 
on the caller’s symptoms, would otherwise advise 
the caller to see their doctor within four hours or go 
to the emergency department. However, we noted 
that about 12% of pages to the on-call physician in 
2008 were for less serious situations, such as the 
caller asking to speak with the physician. In cases 
where a physician is being paged, the nurse advises 
callers that either a nurse or the on-call physician 
will call them back with further instructions within 
30 minutes, but that they should go to the emer- 
gency department if they do not hear back. 

In 2008, over 20,000 calls (that is, roughly one in 
five calls) to THAS resulted in pages to on-call phys- 
icians. In over 70% of these cases, the on-call phys- 
ician had to be paged more than once, and in 9% 
of cases, the pages were not returned. For returned 
pages, 81% were received within 30 minutes. We 


noticed that about 10% of family practices did not 
return at least one-third of the pages they received 
in 2008. There is no financial penalty to the family 
practice if a page is not returned within a reasonable 
time frame or is not returned at all. As well, many 
of the pages were returned after the 30 minutes 
originally indicated to the caller, but no information 
was available on the total number of callers who 
received a return call or went to emergency because 
they had not received a call. 


Information Requests 


The service provider maintains an audiotape library 
of various health-related topics that callers may 
access if they are calling for health information 
rather than health advice, as well as a list of com- 
munity services available around the province. 
According to the service provider, its medical advis- 
ers do not review the taped health topics because 
the tapes are not used to provide care advice. 
However, the service provider uses its medical 
advisers when new health topics need to be created 
(for example, in response to the listeriosis outbreak 
in Ontario). The service provider also had a process 
for updating its database on community resources. 
We noted that 11% of registered calls related 
to health information requests, and 4% related to 
information about community services. Although 
the service provider’s contract with the Ministry 
requires that a nurse speaks to all callers, we 
noted that over 99% of information requests were 
handled entirely by a nurse, rather than the nurse 
connecting the caller to the audiotape library. As 
well, all requests for information on community 
services were handled by nurses. Another juris- 
diction we spoke with indicated that it does not 
require that nurses speak with callers who are just 
requesting information, in order to free up nurses 
to take symptom-based calls, which may reduce the 
wait times for other callers. 
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Confidentiality 


The Ministry’s contract with the service provider 
requires it to ensure the security and integrity of 
caller information and to keep that information ina 
physically secure location. In addition, the contract 
states that the service provider should restrict 
access to personal information to those who need to 
know in order to provide the service. 

We noted that the service provider had policies 
and practices in place that focused on maintaining 
caller confidentiality and privacy. For example, 
employees required passwords to access computer 
records. As well, the service provider had a policy to 
conduct semi-annual reviews of access privileges to 
ensure that staff have access to necessary informa- 
tion only. Further, commencing December 2008, 
the service provider developed a policy to conduct 
quarterly password assessments to help ensure that 
passwords were appropriately strong (that is, not 
based on the user’s biographical information) in 
accordance with policy, and to conduct semi-annual 
audits of computer devices to ensure that anti-virus 
software is operating as intended on all computers. 
The service provider indicated that it was in the 
process of implementing these policies. 
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The service provider indicated that it does not 





perform any vulnerability or penetration testing 
on its teletriage services’ servers and network 
equipment. However, in May 2008, the Ministry 
conducted a Threat Risk Assessment on teletriage 
services. The assessment reviewed the risks associ- 
ated with extracting and transmitting patient and 
physician data from the Ministry’s data centre to 
the service provider for the THAS program. Issues 
noted included: 

e a lack of separation of call information for 
Ontario’s teletriage programs from similar 
programs run by the service provider for other 
provinces; and 

e a lack of encryption of caller information. 

The Ministry indicated that most of the issues 

had been remedied, and the remaining ones were 
expected to be resolved soon. 
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At the time of our audit, we noted that there was 
no agreement with the vendor providing off-site 
storage of call records and the agreement with the 
vendor providing translation services had not been 
updated since it was signed in 2001. In neither case 
had the service provider signed an agreement with 
these vendors to ensure that they comply with the 
confidentiality and privacy requirements set out 
in the service provider’s 2008 contract with the 
Ministry. These requirements include restricting 
access to caller information and keeping it secure, 
in accordance with the Personal Health Information 
Protection Act. 





RECOMMENDATION 2 


To help ensure that all callers’ questions are 
answered within a reasonable time frame, the 
Ministry should: 

@ ask the service provider to instruct its nurses 
to redirect information requests for phone 
numbers and addresses of community servi- 
ces to non-nursing staff; 

e review alternative ways to promote timely 
physician responses to pages for Telephone 
Health Advisory Service callers, such as 
financial penalties when on-call physicians 
do not respond when paged or financial 
incentives for those physicians who consist- 
ently exceed standards; and 

@ require the service provider to measure the. 
wait time for callers from the time the call 
was initially received for both the live and 
call-back queues. 

As well, to ensure that caller information 
remains confidential: 

@ the service provider should sign agreements 
with its vendors that handle confidential 
caller information, such as those providing 
its translation and off-site storage services, to 
maintain appropriate physical and electronic 
security, in accordance with its contract with 
the Ministry; and 





e the Ministry should ensure that periodic 
vulnerability and penetration testing is com- 
pleted at the service provider to identify and 
correct any security weaknesses. 


The Ministry will review and analyze the feas- 
ibility of the recommended changes to how 
teletriage calls are handled. The review will 
explore options to minimize the number of 
“information only” calls that are directed to a 
nurse at the service provider. 

The necessity of the current number of 
physicians supporting the Telephone Health 
Advisory Service is now being reviewed by the 
Ministry to ensure that this component of the 

service is cost-effective. This review will include 
ways to ensure that calls to physicians are 
responded to on a timely basis. The Ministry will 
also review the recommendation to measure the 
wait times for callers, as well as how calls in the 
call-back queue are managed, to ensure that the 
service provider is handling calls in a sequential 
and fairmanner. _ 

The service provider has now signed a ser- 
vice agreement with the off-site storage vendor. 
As well, the Ministry has required the service 
provider to execute with its subcontractors 
the Personal Health Information Protection Act 
sub-agent agreement, which addresses the con- 
fidentiality of caller information. Further, the 
Ministry will perform periodic vulnerability and 
penetration testing at the service provider to 
identify and correct any emerging weaknesses. 


The Ministry and the service provider reached a 
consensus in September 2009 about the content 
for the sub-agent agreement that meets the 
requirements of the Personal Health Information 
Protection Act (PHIPA). The service provider 
indicated that it is in the process of engaging the 





applicable vendors to sign the PHIPA sub-agent 
agreements. However, the service provider 
noted that it may be necessary to find a transla- 
tion vendor located within Ontario, because the 
current vendor operates in another province and 
may not be able to meet certain provisions of the 
sub-agent agreement, such as maintaining per- 
sonal health information from the service pro- 
vider in a physically secure location in Ontario. 


ADVICE TO CALLERS 
Staffing 


The quality of the advice provided to callers 
depends on the qualifications, experience, and 
training of the nurses providing the advice. As of 
December 31, 2008, the call centres were staffed 
with almost 300 nurses, including supervisors and 
site managers. Nurses had an average of four years 
of work experience with the service provider. 

Although the service provider indicated in the 
proposal that it submitted to the Ministry in 2007 
to secure the contract that its nurses have at least 
three years of any type of nursing experience, its 
policies require nurses to have only one to three 
years of clinical experience. From our sample of 
nurses hired in 2008, we noted that 23% had less 
than one year of acute-care experience, and 20% 
had less than three years of total nursing experi- 
ence. Four other provinces with teletriage services 
indicated that their nurses were required to have at 
least three years of acute-care experience. 

We noted that new nurses participate in initial 
classroom orientation training and one-on-one 
training. As well, every nurse is required to partici- 
pate in ongoing training activities, including one 
case study based on common call scenarios and 
four coaching sessions per month. Although docu- 
ments at the service provider indicated that nurses 
completed the required initial training and most of 
the ongoing training in 2008, some team managers 
often did not complete their monthly case studies. 
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In particular, we noted that over 25% of the team 
managers had completed less than half the required 
case studies in 2008. 


Compliance With Clinical Guidelines 


Nurses use decision support software, with medical 
algorithms, which provides them with guidelines 
and protocols for handling symptom-based calls. 
The clinical guidelines and protocols are updated 
annually by two U.S.-based medical doctors, and are 
reviewed by the service provider’s medical advis- 
ers to ensure that they are consistent with medical 
practices in Ontario. The service provider also has a 
process in place for updating the clinical guidelines 
between the annual reviews, if the need arises. 

In the 2008/09 fiscal year, 85% of callers to 
teletriage services were seeking advice for specific 
medical symptoms. Teletriage nurses obtain infor- 
mation from each caller on the nature and severity 
of the health symptoms he or she is calling about. 
This information is entered into a medical algo- 
rithm, which indicates the advice to give the caller. 
The nurses advise callers based on both the medical 
algorithm’s clinical guidelines and their clinical 
judgment. Figure 8 shows the most common advice 
given to callers in the 2008/09 fiscal year. 

The clinical guidelines used by the teletriage 
nurses are intended to provide quality patient care 
advice and to result in the most appropriate use of 





Figure 8: Nurses’ Advice To Callers, by Percentage of 
Calls Answered, 2008/09 


Source of data: Service provider 
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health-care services. According to the service pro- 
vider, in the 2008/09 fiscal year, nurses’ judgement 
led them to deviate from the clinical guidelines for 
5% of callers requesting advice for their symptoms. 
However, for almost 30% of the deviations, the 
reason for the nurses’ judgement was not included 
in the call documentation. As well, the service 
provider’s call audit process noted that, based on 
both the clinical guidelines and the nurses’ clinical 
judgement, the advice provided to about 95% of 
callers was appropriate. Similar results had been 
noted in previous years. 


Callers’ Compliance 


One of the objectives of both Telehealth Ontario 
and THAS is to facilitate the use of the most appro- 
priate health services by consumers. Although there 
are cost savings to the health system from directing 
callers away from an unnecessary visit to the emer- 
gency department, there are also health benefits 
for persons requiring emergency care who had not 
originally planned to go to the emergency depart- 
ment. Therefore, the service provider asks callers 
what they would have done if they had not called 
for advice (for example, self-treat, see a physician, 
go to the emergency department). 

A 2004 study by the Institute for Clinical Evalua- 
tive Sciences examined changes in the utilization 
rates at emergency departments following the 
implementation of Telehealth Ontario, and found 
no significant impact on emergency department util- 
ization rates in five of the six regions it examined. 

In the sixth region, the study noted slightly higher 
emergency department utilization rates following 
Telehealth Ontario’s implementation. The study 
did not look at, amongst other things, the quality of 
care or clinical outcomes resulting from Telehealth 
Ontario. It also indicated that further research 
would be needed to assess these. At the time of our 
audit, no additional analysis had been done. 

We analyzed the data documented by the service 
provider on callers’ initially planned actions com- 
pared to the advice they were given by the nurse. 


For our analysis, we considered the levels of care, 
from lowest to highest, to be self-treat, see a doctor, 
or go to the emergency department. We noted that 
38% of callers were advised to use the same level of 
care as they originally planned, 33% were advised 
to use a lower level of care, and 29% were advised 
to use a higher level of care. Overall, about the same 
percentage of callers were advised to go to the emer- 
gency department as had originally intended to go 
there, but many of the callers who were referred to 
the emergency department were not the ones who 
had originally planned to go. 

The service provider also asks callers whether 
they plan to follow the nurse’s advice. In the 
2008/09 fiscal year, 94% of callers indicated that 
they intended to comply. Since callers are not asked 
to provide their Ontario health card number to the 
service provider, it is difficult to confirm whether 
callers actually follow the nurses’ advice. If the 
health card number were requested, it would be 
possible to check Ontario Health Insurance Plan 
records to determine whether the caller followed 
the advice given. A study (released in 2002) of one 
region in Ontario indicated that actual compliance 
was lower than callers’ self-reported compliance. 
As well, a 2006 Alberta study indicated that only 
75% of callers went to emergency when advised to 
go, and only 47% saw a physician within 24 hours 
when advised to do so. 


RECOMMENDATION 3 





To better ensure that callers to teletriage ser- 
vices receive and follow the most appropriate 
advice to address their health concerns, the 
service provider should: 

e@ hire nurses who have at least three years of 
nursing experience, including at least one 
year of acute-care or clinical experience, in 
accordance with its proposal to secure the 
contract to provide teletriage services and its 
internal policies; 

@ ensure that nurses complete their ongoing 
training in accordance with policies; and 


@ require nurses to document the reason for 
providing advice that does not follow a clin- 
ical guideline or protocol. 

As well, to better determine the impact of 
the advice provided to callers, the Ministry, in 
conjunction with the service provider, should 
develop a process (such as obtaining Ontario 
health card numbers and following up ona 
sample of the callers’ subsequent actions) for 


periodically assessing the extent to which callers 


follow the nurses’ advice. 


Due to the nursing shortage in Ontario, the 
service provider has reviewed and redesigned 
its nurse recruitment and retention strategy. 
The service provider is now testing candidates 
to ensure that they possess critical thinking 

and clinical skills, which it noted were a better 
indicator of clinical competence than years of 
service. Further, in order to be hired by the 
service provider, the candidate must exceed an 
established score, which the service provider 
determined by testing its existing high perform- 
ers who had at least three years of experience 
when hired. The service provider indicated that, 
although these changes have made a positive 
impact on its recruitment and retention efforts 
and have also increased quality outcomes, the 


availability and retention of nurses, regardless of 


years of experience, continue to be a challenge. 
The service provider also noted that it 


employs nurses who work from home in order to 


minimize nurse turnover and mitigate the risks 


of call centre closures associated with a potential 


pandemic crisis. However, the service provider 
commented that a more aggressive work-from- 
home model would help ensure the uninter- 
rupted supply of teletriage services. In the 
future, the service provider envisions a virtual 
telehealth call-centre environment throughout 
the province, which will increase the provider’s 
ability to recruit high-quality nurse applicants 
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province-wide (because nurses do not have to 
live close to a call centre), reduce the level of 
nurse turnover, and provide a stronger business- 
continuity plan for disaster preparedness. 

With respect to nurses completing their 
ongoing training, the service provider has 
reviewed its current policies requiring monthly 
case studies for team managers and determined 
that an alternative approach would be more 
appropriate. The new training requirements are 
expected to be implemented by January 1, 2010. 

The service provider indicated that software 
changes, scheduled for completion by Septem- 
ber 30, 2009, will make it mandatory for nurses 
to indicate the reason for not following the clin- 
ical guidelines before they can proceed further 
in the call. This ensures improved compliance 
with call-handling processes and improved 
accuracy in documenting the reason supporting 
the nurse’s clinical decision to deviate from a 
clinical guideline that did not accurately reflect 
the circumstances. 

The service provider noted that having to 
obtain Ontario health card numbers from every 
caller would increase its time handling each call 
because of the additional time needed for callers 
to locate their health cards. As well, the service 
provider indicated that obtaining the health 
card number must be optional to accommodate 
callers who wish to remain anonymous or can- 
not access an Ontario health card. 


The Ministry will ensure that the requirements 
established in the agreement with the service 
provider regarding the experience and ongoing 
training of teletriage nurses are adhered to. As 
well, the Ministry will ensure that requirements 
regarding the documentation of reasons for 
providing advice that does not follow a clinical 
guideline are adhered to. 

The Ministry will work with the service pro- 
vider to research and review ways to determine 
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the impact of the advice provided to callers. 

The possible collection of Ontario health card 
numbers will be reviewed to determine the 
impact on call time. In addition, because health 
card numbers are personal health information, 
their collection and use by the Ministry for the 
purpose of determining the impact of the advice 
provided to callers will require in-depth analy- 
sis, including an assessment of the protection of 
an individual’s right to privacy. 


QUALITY ASSURANCE 


The service provider has developed a quality assur- 
ance program to monitor and revise the delivery 

of teletriage services. Components of the program 
include periodic review of the nurses’ handling of 
calls, conducting caller satisfaction surveys, follow- 
ing up on complaints, and monitoring by a Quality 
Service Committee at each call centre location. The 
committees meet several times a year, and their 
members include, among others, a medical adviser, 
the manager responsible for clinical activities, the 
site educator, and a nurse. 


Call Audits 


The service provider has a call audit process that 
involves having reviewers listen to at least 15 calls 
per month for each call centre. The reviewers are 
generally selected by the Quality Service Commit- 
tee at each call centre and often are either a clinical 
team manager or a senior nurse. The reviewers 
select which calls to audit and evaluate the advice 
provided by the nurse, as well as the nurses’ 
clinical judgment and communication skills. They 
also ensure that the call information documented 
reflects what transpired. If improvements are 
required, the clinical team manager performs the 
appropriate follow-up with the nurse. 

Results of the audits are summarized monthly 
and discussed by each call centre’s Quality Service 


Committee. The Quality Service Committees 

are responsible for making recommendations to 
address issues noted and for ensuring implementa- 
tion of those recommendations. 

We reviewed the call audit process and noted 

the following: 

e As noted earlier, unlike calls to 911, calls to 
Ontario’s teletriage services are generally 
not recorded. Therefore, reviewers can only 
monitor calls as they take place and when 
call volumes permit. As a result, most of the 
call audits Gncluding 84% of the call audits 
we sampled) are performed during off-peak 
periods. Consequently, call audit results may 
not be indicative of performance during peak 
periods, when nurses are under pressure to 
respond to waiting callers within established 
time frames. 

e Reviewers audit the calls of nurses who work 
at the same call centre that they do. As well, 
the nurse is sometimes the reviewer’s sub- 
ordinate, which means that poor performance 
by that nurse could reflect negatively on the 
reviewer. To determine whether reviewers 
were objective, in 2005 the service provider 
performed a one-time inter-site audit in which 
reviewers audited calls at another call centre 
as well as at their own. These audits indicated 
that reviewers generally rated nurses in their 
own call centre better than nurses in other 
call centres. For example, one call centre 
received a score of 87% from its own reviewer, 
but scored only 74% when evaluated by a 
reviewer from a different call centre. The ser- 
vice provider has not had reviewers audit calls 
at other call centres since 2005. 

@ Reviewers may not evaluate calls in a consist- 
ent manner. On a semi-annual basis, the 
reviewers at each call centre are all expected 
to review the same series of calls to help 
ensure that they will evaluate all calls ina 
consistent manner. We noted that these semi- 
annual evaluations were not done in 2008 at 
the two call centres that handle the most calls. 


e During 2008, at one call centre, no call 
audits were conducted for three months and 
only five audits a month were conducted for 
another three months. We were informed that 
this situation had since been corrected. 

e The service provider does not periodically 
analyze the results to determine whether 
there are any trends or systemic deficiencies 
in the call audit process or the quality of the 
advice provided. ) 

If calls were taped, as is the practice of several 
other provinces we contacted, calls made during 
peak periods could be audited during less busy 
times of the day, and could be more easily audited 
by reviewers from other call centres to ensure a 
more objective evaluation. 

One Canadian jurisdiction that runs a similar 
teletriage service informed us that it uses mystery 
callers on a regular basis to monitor the timeliness 
and quality of the teletriage services provided. 
Mystery callers place calls to the teletriage services 
with predetermined questions, and assess the 
appropriateness of the information and advice 
provided. The Ministry indicated that it does not 
use mystery callers. Most of the provinces we spoke 
with indicated that they tape all calls, so that they 
can be reviewed at a later date to ensure appropri- 
ateness, among other things. The Ministry’s 2003 
internal privacy impact assessment noted that calls 
should generally not be taped. In the absence of 
calls to Ontario’s service provider being taped for 
periodic review, mystery callers could provide some 
assurance on the quality of the teletriage services. 
Alternatively, the Ministry could consult with the 
Information and Privacy Commissioner’s Office 
regarding whether calls can be randomly taped for 
quality assurance purposes. 


Caller Satisfaction Surveys 


Receptionists at the service provider telephone 
selected callers to determine, among other things, 
their satisfaction with the advice they received from 
a teletriage service’s nurse within the last 48 hours. 


Teletriage Health Services 


Most days, the computer generates a list of 150 
eligible callers province-wide, and the receptionists 
survey as many callers as time permits that day. To 

be eligible for the survey, callers must have given 
consent during their original call and must not have 
been surveyed in the previous six months. Callers who 
respond negatively to certain questions may be con- 
tacted again by a clinical team manager at the service 
provider, to ensure that their concerns are addressed. 
In 2008, about 9,000 callers were surveyed. 

A monthly summary of satisfaction survey 
results is reviewed by each call centre’s Quality 
Service Committee, which recommends remedial 
action if warranted. In addition, every month the 
service provider reports to the Ministry the number 
of callers surveyed and the overall satisfaction rate. 

We reviewed the results of caller satisfaction 
surveys completed by the service provider from 
2006 through 2008 and noted a high satisfaction 
rate with the overall service. On average, 98% of 
callers surveyed reported that they were satisfied 
with the service and would use the service again if 
the need arose. 

Because the satisfaction surveys are conducted 
by the service provider, our independent survey 
asked people who had previously called Telehealth 
Ontario about their satisfaction with the service. 
The results of our survey indicated that overall, 
82% were either somewhat satisfied or very satis- 
fied with the support and advice they received. 
However, almost 30% of Northern Ontario resi- 
dents were either somewhat dissatisfied or very 
dissatisfied. This result should not be interpreted 
to mean that this issue is necessarily related to 
the northern call centre, because calls get routed 
to the call centre that can answer the fastest. Our 
survey did not inquire about the reason for callers’: 
dissatisfaction. 

We noted that the service provider does not sur- 
vey other stakeholders, such as family physicians 
and emergency departments. In 2005, the Ministry 
commissioned a survey of family physicians and 
emergency department staff regarding THAS. 
About 50% of responding physicians indicated that 
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THAS resulted in more appropriate emergency 
department use by patients, although 15% believed 
that THAS referred too many patients to the emer- 
gency department. No similar survey has been 
completed for Telehealth Ontario. 

We noted that other provinces periodically 
survey health-care providers to determine their 
awareness of the province’s teletriage services as 
well as their opinion about the value of the services. 
The jurisdictions we spoke to indicated that health- 
care professionals’ support and acceptance of the 
teletriage services was crucial to the success of their 
programs. 


Complaints Process 


Complaints concerning teletriage services may be 
made to either the Ministry or the service provider. 
Complaints received by the Ministry may be followed 
up directly or, with the complainant’s permission, 
forwarded to the service provider for follow-up. 
During 2008, the Ministry logged only 12 complaints 
and the service provider logged 658 complaints. 

Both the Ministry and the service provider have 
processes in place for handling complaints. We 
reviewed the complaints handling process at the 
Ministry and noted that the appropriate action was 
generally taken. 

We also reviewed the complaints handling 
process at the service provider. We noted that com- 
plaints were generally about the attitude of nurses 
(26%), the length of the wait for a call-back (20%), 
and the quality of care advice (19%). According 
to the service provider, resolving a complaint took 
an average of 15 days in 2008, with 90% of the 
complaints being resolved within 30 days. Most 
complaints we sampled were investigated by senior 
staff, in accordance with the service provider’s poli- 
cies. However, because calls were not taped, it was 
not possible to know exactly what transpired during 
the calls: a call’s documentation may reflect only the 
nurse’s perception of events. Understandably, nurses 
may have difficulty recalling a specific call, because 
they handle about 30 calls per eight-hour shift. 


Therefore, it was not possible to determine whether 
complaints were appropriately followed up on. 


Quality Service Committees 


As noted earlier, the service provider has a Quality 
Service Committee at each call centre to monitor 
and evaluate certain quality indicators, and recom- 
mend remedial action if needed. 

The quality indicators evaluated by the commit- 
tees include the following: 

e call volume statistics; 

e call duration and call-back time statistics; 

e disposition of calls by type of advice; 

e risk management outcomes, such as number 
and overall result of call audits, number of 
complaints, unusual incidents, and results of 
caller satisfaction surveys; and 

e staff development, such as the number of 
coaching and training sessions. 

We reviewed the Quality Service Committee’s 
minutes for each site for the three years ending 
December 2008 and noted that they did identify 
certain issues. We were informed that follow-up 
action was taken, but due to a lack of documenta- 
tion, we could not always tell whether the issues 
were resolved. As well, we noted that although call 
audits and complaints were being accumulated 
province-wide, there was no documented trend 
analysis by call centre or by nurse. 


- RECOMMENDATION 4 





To better ensure the quality of teletriage services 
and identify areas for improvement: 

e the service provider should have independ- 
ent reviewers conduct an established 
number of random audits on calls received 
at different times of the day and on different 
days of the month, including weekends and 
holidays; 

@ the service provider should periodically 
analyze the overall issues noted in call audits 
and complaints byxcall centre and by nurse 


to determine whether there are any systemic 
issues or trends that warrant follow-up; and 

@ the Ministry should conduct periodic 
independent satisfaction surveys of indi- 
viduals impacted by teletriage services, 
including callers, physicians, and emergency 
department staff. 

The Ministry should request the Information 
and Privacy Commissioner’s input on whether 
calls to the service provider can be taped for per- 
iodic review to determine the appropriateness 
of advice provided by teletriage nurses. If calls 
are not taped for periodic review, the Ministry 
should seek another way to obtain independent 
assurance on the appropriateness of advice pro- 
vided by teletriage nurses (for example, through 
the use of mystery callers). 





The service provider commented that peak 


periods or periods of unusually high call volume 
require “all hands on deck,” meaning every 
employee capable of getting on the phones 

at each site does so to manage the incoming 
call volumes. Therefore, calls during these 
periods are not monitored because reviewers 
are handling calls, and the service provider is 
not permitted to record these calls for review 
at a later time. The service provider indicated 
that it would be beneficial to record calls 
because they could be reviewed at a later time 
by an independent person. This would improve 
quality assurance and customer service when 
responding to complaints from any source, 
including the public and doctors. Further, the 
service provider recognizes the limitations of 
the current system for the selection of calls to 
be audited and is considering using computer- 
generated reports to randomly select these calls; 
however, this hinges on implementation of an 
automated system to record calls. 
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The service provider noted that issues aris- 
ing from call audits and complaints are being 
reviewed for any issues or trends at each call 
centre, as well as province-wide monthly, with 
action taken when appropriate. However, these 
reviews are currently not documented. The 
service provider intends to perform monthly 
trend analyses on the issues arising from call 
audits and complaints, identify the contributing 
factors, develop an action plan, and communi- 
cate this to the appropriate individuals. Trend 
analysis will be completed on an individual, 
team, site, and province-wide basis and on 
parts of the call process where there are oppor- 
tunities for improvement in call quality that 
would facilitate positive patient outcomes. The 
service provider would also like to implement 
an improved quality-services program that util- 
izes quality-services associates who would be 


independent of operations and sites to randomly 


audit the quality of calls 24 hours a day, seven 
days a week. This would increase the ability to 
identify and analyze trends without bias. This 
type of program would require an automated 
call-recording system. 





The Ministry will review and amend the agree- 


ment with the service provider, as necessary, 

to improve the quality-assurance process to 
identify any evolving systemic issues and trends 
related to teletriage services. The Ministry will 
also plan for and conduct, at appropriate inter- 
vals to ensure meaningful feedback, independ- 
ent satisfaction surveys of individuals affected 
by teletriage services. 

The Ministry was previously informed that 
taping calls was inadvisable. However, the 
Ministry will revisit this issue and consider the 
use of mystery callers to ensure that all aspects 
of the teletriage services can be monitored and 
managed as necessary. 
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PAYMENTS FOR TELETRIAGE SERVICES 


The service provider was awarded the most recent 
teletriage services contract based on its response 
to a public procurement process, which resulted 

in its being evaluated as having the best bid of the 
three bids submitted. Under the contract, which 
was effective April 1, 2008, the service provider 

is paid based on the number of registered calls. 
Registered calls are those calls where a nurse 
provided advice or information to a caller, or tried 
unsuccessfully three times to reach a caller who 
had requested a call-back. For the 2008/09 fiscal 
year, the service provider was paid $35.1 million for 
the teletriage services: a flat fee of $35 million for 
the first 900,000 registered calls (or about $39 per 
registered call) and about $27 per registered call 
after that. 

We noted that, as expected, the average cost 
per call handled by a nurse at the service provider, 
and paid to the service provider, was less than 
the approximately $56 that physicians earn if the 
patient visits them in their office, or the approxi- 
mately $98 physicians earn if they see a patient 
in the emergency department. However, three of 
the other provinces that shared cost information 
with us indicated that their cost per call was about 
$20. The Ministry had not investigated the reasons 
underlying the significant difference in costs per 
call between Ontario and other jurisdictions, but 
possible explanations could include different cost- 
ing methodologies, such as not including all capital 
costs, and variances in nurses’ salaries. 

To help ensure that the service provider is billing 
the Ministry for the correct number of calls, the 
Ministry reviews reports on call volumes provided 
by the service provider. During the 2008/09 fis- 
cal year, the service provider noted that it had 
incorrectly billed the Ministry for a number of 
months. The errors were brought to the Ministry’s 
attention and corrected. To prevent similar prob- 
lems in the future, in December 2008 the Ministry 
entered discussions with the service provider to 
obtain data on all calls. With this information, the 


Ministry can verify the number of calls submitted 
for payment. At the time of our audit, discussions 
for the secure transfer of this information were 
ongoing, although a date to commence the transfer 
of information had yet to be finalized. 

During the 2008/09 fiscal year, the Ministry also 
paid $900,000 to the Ontario Pharmacists’ Associa- 
tion (OPA) for calls to its Medication Information 
Service. The payments are made according to a pre- 
determined budget that is based primarily on call 
volume and is approved by the Ministry. We noted 
that in 2008 the OPA reported receiving about 
double the number of calls that the service provider 
said it made to them. We asked the Ministry about 
this during our audit, and it advised us at that time 
that it would investigate this difference. 





RECOMMENDATION 5 


To ensure that the amount paid for teletriage 
services is reasonable in comparison to other 
jurisdictions and in accordance with the Min- 
istry’s contract with the service provider, the 

Ministry should: 

e obtain information on the delivery of teletri- 
age services in other provinces to determine 
whether there are areas where Ontario’s 
teletriage services could be delivered more 
economically; and 

e confirm that payments made to the Ontario 
Pharmacists’ Association’s Medication Infor- 
mation Service are reasonable, based on the 
actual number of calls that the Telehealth 
Ontario service provider reports having 
referred to the Medication Information 
Service. 





The 2007 procurement process for teletriage 


services provided assurance that the amount 
being paid for the services is competitive within 
the Ontario market. 


The Ministry is aware that there are differ- 
ences in the way teletriage services are provided 
across the country and, therefore, possible varia- 
tions in the way the services are costed. Ministry 
staff will consult with their provincial counter- 
parts to determine what their cost per call rep- 
resents and whether there are opportunities to 
deliver Ontario’s services more economically. 

The Ministry is satisfied that the payments 
made to the Ontario Pharmacists’ Association 
(OPA) are correct based on the calls reported by 
the OPA. The Ministry confirmed that the actual 
number of calls the service provider reported as 
having referred to the OPA’s Medication Infor- 
mation Service was incorrect. Actions are being 
taken by the service provider to ensure that the 
number of calls reported being referred to the 
Medication Information Service is correct and 
the Ministry will ensure that ongoing reporting 
from the service provider and the OPA are 
consistent. 


EFFECTIVENESS OF TELETRIAGE 
SERVICES . 


The Ministry obtains information on a regular basis 
from the service provider. Monthly, the Ministry 
receives information on various items, such as 
achievement of the key performance standards, 
number of callers, caller demographics, caller satis- 
faction, and the most common symptoms for which 
advice was sought. On a quarterly basis, the Min- 
istry receives information on other items, including 
caller acceptance of advice and any deviation of a 
nurse’s advice from the clinical guidelines. Other 
provinces with teletriage services that we spoke to 
indicated that they generally collect similar infor- 
mation, although there were some variances (for 
example, one province obtained monthly informa- 
tion on all call audits). 
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Key Performance Standards 


The Ministry’s contract with the service provider 
requires that specified key performance standards 
be met for the teletriage services. If these standards 
are not met, the service provider incurs financial 
penalties. As shown earlier in Figure 5, the service 
provider met the standards for the 2008/09 fiscal 
year. Although the standards focus primarily on 
access to services, there are no performance stan- 
dards relating to callers waiting in the live queue. 
As well, similar to other Canadian jurisdictions we 
spoke with, there are no standards with respect to 
the quality of nurses’ advice. 

With respect to abandoned calls, the service 
provider indicated that callers who hang up dur- 
ing the recorded greeting usually have called the 
wrong number or have changed their mind about 
using the service. Callers who end the call or hang 
up later in the call process (that is, after a call was 
answered by the receptionist or the voice message 
system) include those who do not wish to wait 
for a call back from a nurse and those who tire of 
Waiting in the live queue to speak to a nurse. We 
noted that the contractual definition of abandoned 
calls excludes these callers. However, we noted 
that if callers who did not wish to wait for a call 
back or tired of waiting in the live queue were also 
included, the total rate of abandoned calls to the 
teletriage services during the 2008/09 fiscal year 
would increase from about 2% to almost 17%. 
Furthermore, according to the service provider, 
about 25% of callers waiting in the live queue 
hung up before the call was answered by a nurse. 
Information was not readily available to enable us 
to confirm this number. 


Ministry Performance Measures 


In order to measure achievement against the stated 
program objectives, the Ministry implemented 
performance measures for Telehealth Ontario in the 
2005/06 fiscal year. Since then, the Ministry has 
reviewed and updated the performance measures 
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annually, and introduced similar measures for THAS. 
These are good initiatives, although we did note that 
this information is generally not reported publicly. 

As shown in Figure 9, results for three of the 12 
performance measures were not available in the 
2008/09 fiscal year. According to the Ministry, a 
survey of the general public would be required to 
collect this information. As previously mentioned, 
the last such survey conducted by the Ministry, in 
2006, indicated that about two-thirds of Ontario 
residents were aware of Telehealth Ontario. How- 
ever, no questions were asked regarding whether 
callers believed that the advice received improved 
their health education or ability to take the most 
appropriate health-care action. 

Although performance measures exist for all pro- 
gram objectives, there are no indicators to address 
the quality of the advice provided. Such measures 
could include the percentage of call audits where 
the nurse’s advice was found to be inappropriate, 
and the percentage of callers who make repeat calls 
for the same symptoms. Other provinces indicated 
that they used some similar performance measures, 





but the measures varied. For example, some other 
provinces had performance measures for average 
call length and for the percentages of callers who 
are given various categories of advice (i.e., go to 
emergency, see their doctor at the next possible 
opportunity, or conduct self-care). 


Program Evaluations 


Telehealth Ontario 
Between 2001 and 2006, various reviews of 
Ontario’s teletriage services were conducted. In 
particular, the Ministry commissioned an external 
consultant to conduct an evaluation of Telehealth 
Ontario, from April 2003 to June 2005, at a total 
cost of $912,000. This evaluation was intended to 
assess the program’s performance relative to its 
objectives; identify opportunities to increase its 
effectiveness; and assess its overall impact on the 
health-care system in Ontario. The evaluation, 
which involved a series of studies that were based 
on caller surveys, concluded that the program 
had been effective at directing callers to the most 


Figure 9: Performance Measures and Targets (Where Established) for Teletriage Services, 2008/09 


Source of data: Ministry of Health and Long-Term Care 


Objective 
facilitate the use of the most 








Performance Measures and Associated Targets 
% of callers who were re-directed to a higher level of care 





Achievement 


heehee 29% 
appropriate health services than their original intent 

% of callers who were re-directed to a lower level of care 33% 

than their original intent ; 

% of callers who intend to comply with nurse’s advice 94% 


improve access to appropriate health 
information and advice 





% of Ontarians who are aware of the service 


% of population of each Local Health Integration Network 
(LHIN) utilizing the service 


not reported 





not reported 


80% of incoming calls to be answered by a live voice 97% 
85% of callbacks to be made within 30 minutes 88% 
98% of callbacks to be made within two hours 99% 
Rate of abandoned calls to be less than 6% 2% 


increase health education and improve 
decision-making 

improve satisfaction with access to 
quality health information and access 
to appropriate health services 


% of callers who report increased confidence in health- 
Care decision-making and administering self-care 


not reported 


% of callers who are satisfied with the service 98% 


% of negative feedback received 0.1% 
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appropriate level of care. As well, the evaluation 
indicated that, among other things, the percentage 
of repeat calls regarding the same symptoms had 
declined, indicating that nurse advice had become 
more appropriate. 


Telephone Health Advisory Services 
Although the Ministry has not recently reviewed the 
operations of THAS, in the 2004/05 fiscal year the 
Ministry commissioned an external consultant, at a 
total cost of $127,000, to conduct a series of studies 
on THAS. These studies included an assessment of 
the program’s performance; an assessment of the 
program’s impact on patients, health-care providers, 
and the health system; and a comparison of the esti- 
mated cost of caller intentions before calling THAS 
and their intended actions after calling THAS. 

The studies reached favourable conclusions on 
the program, showing a high level of satisfaction 
among callers and participating physicians. As well, 
the cost analysis of THAS indicated that almost 
$90,000 was saved per 100,000 calls. The studies 
also noted that the proportion of callers who were 
advised to go to the emergency department was 
similar to the percentage of callers who had had 
that intention before calling, but many of the call- 
ers who were referred were not the ones who had 
originally planned to go. 
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RECOMMENDATION 6 


To better ensure that teletriage services are 
meeting their objectives, the Ministry, in 
conjunction with the service provider, should 
expand the performance standards to include 
indicators on callers who wait in the live queue 
(including how long they wait and how many 
hang up before speaking to a nurse) and on the 
quality of the nurses’ advice. 

As well, because it has been almost five 
years since the effectiveness of the teletriage 
services in meeting their established objectives 
has been assessed, the Ministry should consider 
conducting a formal evaluation. One area 
to consider including in the evaluation is an 
assessment of whether using a teletriage service 
improves callers’ health-related decision- 
making. 


The Ministry will work with the service provider 
to develop effective performance measures 
regarding callers who wait in the live queue. In 
addition, the ability to measure—and possible 
methods of measuring—the quality of nurses’ 
advice will be investigated. 

Further, the Ministry will conduct a formal, 
external evaluation of teletriage services to 
measure the effectiveness of the program 
against its identified objectives. 
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Background 


The Workplace Safety and Insurance Board (WSIB) 
is a statutory corporation created by the Workplace 
Safety and Insurance Act, 1997 (Act). Its primary 
purposes are to provide income support and fund 
medical assistance to workers injured on the job. 
The WSIB also funds programs to help prevent 
workplace injuries, illnesses, and fatalities. The 
WSIB endeavours to apply an integrated approach 
to workplace health and safety that promotes co- 
operation and collaboration among its stakehold- 
ers, including the government. 

The workplace safety and insurance system is 
financed through premiums charged on the insur- 
able payrolls of employers; the WSIB receives no 
funding from the government. Under the Act, the 
government has the sole responsibility for setting 
benefits and coverage through legislation, while the 
WSIB has direct responsibility for setting premium 
rates, within the following guideline: 


The Board has a duty to maintain the 
insurance fund so as not to burden 
unduly or unfairly any class of Schedule 1 
employers [generally all private-sector 
employers] in future years with payments 
under the insurance plan in respect of 
accidents in previous years. 


em) Unfunded Liability of the 
Workplace Safety and 
Insurance Board 


Notwithstanding this legislative guideline, the 
assets in the WSIB insurance fund are substantially 
less than what is needed to satisfy the estimated 
lifetime costs of all claims currently in the system— 
thus producing what is known as an “unfunded 
liability.” 

In our 2005 Annual Report, we noted that the 
WSIB’s unfunded liability had reached $6.4 billion 
at that time, and commented on the importance of 
the WSIB having a credible plan to reduce it. We 
noted that failure to effectively control and elimin- 
ate the unfunded liability could result in the WSIB 
being unable to meet its existing and future finan- 
cial commitments to provide worker benefits. 

We decided to revisit our previous comments on 
the unfunded liability with a view to providing a 
more detailed commentary on the issue given the 
recent turmoil in the global financial markets and 
the impact this has had on the viability of pension 
plans and other worker benefit plans, such as work- 
ers’ compensation insurance. 

Workplace safety and insurance systems operate 
in a complex business environment because they 
serve a number of stakeholders with competing 
interests and views pertaining to the key areas of 
insurance benefits, coverage, and premium rates. 
For instance, employers want low premium levels 
while workers want high benefit-payment levels. 
These competing interests influence benefits, cover- 
age, and premium rates, which can have a negative 
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impact on the size and growth of the unfunded lia- 
bility. It is incumbent on the WSIB and the govern- 
ment to try to balance such views against the need 
to maintain financial stability. 


Review Objective and Scope 





Our work focused on providing information on the 
changes in the unfunded liability, the factors con- 
tributing to these changes, and the initiatives being 
undertaken by the WSIB to control the growth of 
the unfunded liability. We did not audit the WSIB’s 
finances or controls because these aspects are 
examined annually by other auditors. 

We caution the reader that references are made 
throughout our report to future unfunded liabil- 
ity estimates, which are based on assumptions 
regarding future events. Actual results may of course 
vary significantly from these estimates. As well, the 
degree of uncertainty will generally increase the fur- 
ther into the future the estimates extend. We have 
not audited the data the WSIB provided to us and do 
not express an opinion on the actuarial assumptions 
it has made or the methods used. 

As well as conducting work at the WSIB, we 
approached four of the larger workers’ compensa- 
tion boards in other provinces to discuss the prac- 
tices and actions that these jurisdictions have put 
in place to allow them to achieve and/or maintain 
over the last 10 years their fully funded position 
(that is, their assets are sufficient to cover their lia- 
bilities). Two boards responded to our requests; we 
visited these boards to discuss with their officials 
some of the common issues and their approaches 
to managing them. Our observations on some of 
the practices implemented by the other provincial 
boards to support the financial sustainability of 
their systems are also presented in this report. 

We also met with the WSIB’s Chair, its President 
and CEO, members of its senior management team, 
and current and former members of its Board 
of Directors to discuss their perspectives on the 


unfunded liability. We would like to acknowledge 
their assistance during our review. 





The WSIB’s funding ratio represents the percentage 


of assets it has available to meet its financial obliga- 
tions. As of December 31, 2008, its funding ratio 
was 53.5%—significantly lower than any of the four 
large provincial boards with which we compared 
Ontario (British Columbia, Alberta, Manitoba, and 
Quebec), which averaged 102%. In each of these 
four provinces, legislative and policy differences are 
key factors that contribute to their higher funding 
ratios. A Board’s funding ratio is largely determined 
by the size of its unfunded liability, which is the 
amount by which the Board’s financial obligations 
exceed its assets. As of December 31, 2008, the 
WSIB’s unfunded liability was $11.5 billion—an 
increase of $3.4 billion from December 31, 2007. 
One factor that had a significant negative impact on 
the unfunded liability in 2008 was the global eco- 
nomic downturn. However, there are also a number 
of other systemic issues that have affected the size 
of the unfunded liability. 

The main observations arising from our review 


‘are as follows: 


e Eliminating or reducing the unfunded liability 
requires the interaction of four key levers— 
legislated benefits, coverage, premium rates, 
and investments—to work effectively in 
tandem. The inability to eliminate the WSIB’s 
unfunded liability over the last two decades 
has been owing in part to the desire to satisfy 
all the stakeholders. Both the WSIB and the 
government may have to commit to a different 
strategy with respect to the setting of pre- 
mium rates and benefits if the WSIB is to be 
able to eliminate the unfunded liability within 
a reasonable period. 

e The WSIB advised us that its 2008-12 strategic 
plan, The Road to Zero, contains a number 
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of initiatives that target and support the 
financial sustainability of the system. Key 
initiatives include the prevention strategy, 
which provides for a collaborative prevention 
partnership model; social marketing, which 
supports the organization’s prevention man- 
date; development of a new case-management 
approach to service delivery; and other initia- 
tives in the areas of return-to-work programs, 
health-care solutions, and occupational- 
disease services. However, despite these initia- 
tives, the WSIB advised us that, because of the 
significant financial losses resulting from the 
global financial market downturn, its target 

of full funding by 2014, which was originally 
established in 1984, will not be achieved. The 
WSIB has not set a new target date. On the 
basis of February 2009 projected estimates, 
the unfunded liability may not be eliminated 
until 2022—eight years past the 2014 date tar- 
geted by the WSIB and successive governments 
since 1984. 

The WSIB’s actions to eliminate the unfunded 
liability have been limited by recent economic 
circumstances and by the four key levers that 
are to some degree beyond the control of 

the WSIB. These include the government’s 
responsibility and authority over legislation, 
including benefit changes and the extent of 
business sectors and industries that are cov- 
ered by the system. As of 2007, the percentage 
of the workforce covered by the system in 
Ontario was 72.6%, as compared to Alberta at 
89.7%, B.C. at 93.1%, and Quebec at 93.4%. 
The WSIB has the direct responsibility for 
setting premium rates. The WSIB and the 
government face the least resistance from 
stakeholders when they keep premiums low 
(which satisfies employers) and benefits high 
(which satisfies workers). Over time, this can 
result in a large unfunded liability. To mitigate 
against this risk, the provincial boards we 
visited cited examples of legislated require- 
ments or formalized funding models that 


required their systems to be fully funded. We 
recognize, however, that this is a policy issue 
specific to each jurisdiction and that neither 
the WSIB’s legislation nor its Memorandum 
of Understanding with the Ministry of Labour 
requires that the plan be fully funded. 
Premiums have a significant impact on the 
size of the unfunded liability. However, 
annual premium revenues in recent years 
have not been sufficient to cover benefit costs. 
The WSIB has not reported an annual surplus 
since 2001, and since that date the annual 
deficits have ranged from a low of $142 mil- 
lion in 2006 to a high of $2.4 billion in 2008. 
Although investment losses in 2007 and 

2008 have increased the unfunded liability, 
we believe that even if these losses had been 
netted against previous unrealized investment 
gains, the unfunded liability would still have 
been in excess of $7 billion. In the seven years 
since 2001, while the WSIB has experienced 
annual deficits averaging over $900 mil- 

lion, premium rates were only increased in 
2003 and 2006, resulting in an overall 7.5% 
increase or 1% per year on average. Since 
2006, premium rates have not changed to 
reflect the impact of higher benefit costs— 
such as the $750-million benefit enhance- 
ments the government enacted in 2007, which 
did not have matching premium or investment 
revenues to offset the increased costs. 

Benefit and health-care costs have been rising 
over the last 10 years. These cost increases—in 
particular, benefit cost increases arising from 
increases in the amount of time that workers 
are staying on benefits and increases in bene- 
fits arising from legislative changes—have 
contributed to the unfunded liability. Key fac- 
tors identified by the WSIB for the increasing 
duration of claims include some unintended 
consequences of Bill 99, The Workers’ Compen- 
sation Reform Act (which was the last major 
legislative reform to the system), along with 
an ineffective employer-incentive program. As 
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well, an increased use of prescription pain kill- 
ers has added to the costs of the system. 

In the current economic climate, the WSIB 
and the government face significant challenges in 
eliminating the unfunded liability. Increasing pre- 
mium rates would be challenging because Ontario 
already has one of the highest average premium 
rates of any province. The WSIB’s 15-year average 
rate of return on its investments from 1994 to 2008 
was 6.6%, slightly less than the WSIB’s 7% target. 
Given that future benefit costs are expected to rise 
at 7% annually, investments must earn more than 
7% before any reduction of the unfunded liability 
can be realized solely from investment returns. 

On the benefit cost side, the WSIB has undertaken 
a number of internal initiatives to reduce claims 
duration, including the implementation of a new 
service delivery model and the introduction of 
new technology initiatives to reduce health-care 
costs. However, in addition to improved investment 
returns and further cost-reduction measures, more 
significant structural changes, including legislative 
reforms, may be needed to ensure that the Board 
continues to have the ability to meet its future 
financial obligations. 

The WSIB acknowledges that significant actions 
will need to be taken to get its financial affairs in 
order. In spring 2009, the WSIB’s Chair initiated 
province-wide stakeholder consultations on solu- 
tions to the financial challenges facing the WSIB. 
According to the WSIB’s March 2009 newsletter, this 
process was to be “aimed at achieving a broad con- 
sensus among stakeholders on how best to deliver 
a sustainable future for Ontario’s Workplace and 
Safety Insurance Board” and was to “include open 
and frank communication about the financial and 
legislative framework in which the WSIB operates.” 


As the Auditor General’s review notes, the 
growth in the unfunded liability to $11.5 billion 
as of December 31, 2008, is of concern to the 
WSIB. The WSIB faced a key challenge in 2008 


because financial results were significantly 
affected by the global economic downturn. This 
major decline in equity markets led to a signifi- 
cant decrease in investment returns in 2008, 
which in turn resulted in a $3.4 billion increase 
in the unfunded liability in 2008. The ongoing 
market volatility and uncertainty in 2009 con- 
tinues to have an impact, but, to address this 
issue, an enhanced investment strategy was 
implemented in late 2008 to reduce the impact 
of potential market volatility and to better 
align investment assets to long-term funding 
obligations. 

The WSIB’s vision is the elimination of all 
workplace fatalities, injuries, and illness. In 
2007, the WSIB implemented The Road to Zero, 
which is the five-year corporate strategy for 
2008-12. Financial sustainability is one of four 
key fundamentals of The Road to Zero, and, in 
that regard, the WSIB has implemented a fund- 
ing framework and regularly reviews funding 
scenarios and financial results through a process 
of funding outlooks. The WSIB’s prevention 
efforts are already showing positive results: as 
the lost-time injury rate (the lost-time injury 
count per 100 covered workers) has decreased 
from 2.37 in 2000 to 1.51 in 2008, which rep- 
resents an annual rate of change of -4.9% over 
2000-08. 

The WSIB is taking a measured, fiscally pru- 
dent approach in setting premium rates to avoid 
placing undue financial burdens on employers. 
For 2010, premium rates will closely reflect the 
performance of rate groups with poorer health 
and safety performance and other factors, while 
addressing the financial pressure facing the 
WSIB. Only those rate groups with poorer per- 
formance will experience rate increases; other- 
wise, premium rates will remain at 2009 levels. 
Unlike previous years, there will be no premium 
rate reductions for any rate groups, including 
those with improving safety records. 

The WSIB is committed to the goal of long- 
term financial sustainability and expects to 
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establish the new target date to eliminate the 
unfunded liability shortly. The WSIB agrees 
with the Auditor General’s observation that a 
comprehensive, balanced approach is required 
to ensure that the four key levers of legislated 
benefits, coverage, premium rates, and invest- 
ments work in tandem to effectively support the 
elimination of the unfunded liability. 


Detailed Review Observations 


ONTARIO’S WORKPLACE SAFETY AND 
INSURANCE SYSTEM 


Workers’ compensation boards exist in each of 
Canada’s provinces and territories. These organiza- 
tions provide assistance to workers who have been 
injured on the job or have job-related illnesses, and 
also to promote safety awareness with a view to 
preventing workplace injuries. Each organization 
was created under provincial or territorial law. 

The workers’ compensation systems are generally 
similar in structure and mandate, but they are not 
identical in all respects. The provincial organiza- 
tions may differ from each other in size, policies 
concerning the employers, injuries, and illnesses 
they cover, and the benefits they offer. 

One of the key principles adopted when 
Ontario’s workers’ compensation system was estab- 
lished—a principle that still guides the system—is 
that in exchange for guaranteed protection, injured 
workers give up the right to sue employers over 
work-related injuries or illnesses, regardless of 
fault. The legislation created a compulsory no-fault 
insurance and collective liability system adminis- 
tered by a workplace safety/workers’ compensation 
organization—an independent public agency that 
adjudicates the claims of injured workers or their 
survivors and provides compensation where the 
agency considers it appropriate. To help fund the 
benefits and the administration costs, premiums are 


levied on most employers and paid into a fund held 
by the agency. 

The Workplace Safety and Insurance Board 
(WSIB) is a statutory corporation under the Work- 
place Safety and Insurance Act, 1997. Its mandate 
is to promote the prevention of injuries, illnesses, 
and fatalities in Ontario workplaces. The Board 
also provides insurance benefits to the survivors of 
workers who die from a workplace injury or from an 
occupational illness and provides loss-of-earnings 
assistance to injured workers receiving benefits. 

The WSIB is the largest workplace safety and 
insurance/workers’ compensation organization in 
Canada. According to the Association of Workers’ 
Compensation Boards of Canada’s 2007 Annual 
Report, the WSIB is ranked first in Canada in 2007 
by number of claims (about 329,000 compared to 
second-place Alberta with about 175,000 claims), 
and by premium revenue ($3.5 billion compared to 
second-place Quebec with $2.3 billion). 

The WSIB administers the Act for two groups of 
employers: 

@ Schedule 1 employers, which, under a “col- 
lective liability” system, are required to 
contribute to the WSIB’s insurance plan. They 
include, among others, any firms involved in 
the automotive, construction, manufacturing, 
and transportation sectors. 

e@ Schedule 2 employers, which are self-insured, 
are individually liable for the full costs of 
any claims made by workers. They include, 
among others, the provincial government, 
Crown agencies, and some municipalities and 
school boards. The WSIB pays the benefits 
of Schedule 2 workers but is reimbursed by 
the employers for the cost of the claims, for 
administrative costs, and for a portion of the 
cost of the WSIB’s prevention activities. 

The Act requires that the WSIB maintain an 
insurance fund to “pay for the benefits under the 
insurance plan to workers...and to survivors of 
deceased workers” and to pay the WSIB’s operat- 
ing expenses (that is, the cost of administering 
the Act). Under the Act, funding of the system is 
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the responsibility of employers, including govern- 
ment and government agencies as employers in 
Ontario. As in other provinces, Ontario’s system is 
financed through insurance premiums charged on 
the insurable payrolls of employers. The insurance 
premiums paid by the employers vary depending on 
the degree of safety risk of the employer’s type of 
business. The WSIB receives no direct funding from 
the provincial government. 

The WSIB has two sources of revenue: employer 
premiums and investment income. Similar to 
pension plans and other insurance companies, 
investment returns on the fund’s assets are a key 
revenue source for the fund to make future benefit 
payments. 

On the expense side, the WSIB provides bene- 
fits for loss of earnings, benefits for permanent 
impairment, payments for health-care expenses, 
assistance to facilitate an injured or ill worker’s 
return to work, and survivor benefits in the case 
of work-related fatalities. Annual benefit costs for 
accounting purposes consist of two components: 
benefit costs paid (the amount paid to injured 
and ill workers during the year) and changes in 
benefit liabilities (the adjustment to the actuarially 
determined estimates for future claim payments for 
current and prior-year claims). The benefit liabil- 
ities are calculated based on actuarial assumptions. 
Changes in some assumptions can cause significant 
changes in the benefit liabilities. Key actuarial 
assumptions include the discount rate (that is, the 
interest rate used in “discounting”—or determining 
the present value of—future cash flows), mortality 
rates, lost-time injury rates, and inflation factors. 

If premiums collected from employers and 
returns on the WSIB’s investments are insufficient 
to cover the total expected future benefit costs, the 
shortfall will result in an unfunded liability. Put 
simply, the unfunded liability is merely a measure 
of the difference between the value of the WSIB’s 
assets and its estimated financial obligations at a 
point in time. 

The Act does not require the insurance plan 
to be fully funded. The legislative provisions that 


relate to the plan’s funding status stipulate only 
that funds must be “sufficient to make the required 
payments under the insurance plan as they become 
due”. However, the Act does state that employers 
in future years are not to be burdened with “Day- 
ments...in respect of accidents [that occurred] in 
previous years.” 


GROWTH OF THE UNFUNDED LIABILITY 


The WSIB’s unfunded liability was $9.1 billion in 
1990 and stood at $11.5 billion as of December ai 
2008. As Figure 1 shows, there was a significant 
decline in 1997, resulting from legislative reforms 
to the system that arose from the passage of Bill 
99, The Workers’ Compensation Reform Act, which 
revised inflation protection to partially disabled 
workers. During the last three years, from 2006 to 
2008, the unfunded liability almost doubled in size, 
growing from $5.9 billion on December 31, 2006, 
to $11.5 billion—one of the highest levels in the 
WSIB’s history—on December 31, 2008. 

The funding ratio (the ratio of assets to lia- 
bilities) is a useful measure of the adequacy of a 
workers’ compensation system to pay future claims. 
A ratio above 100% indicates that the workers’ 
compensation organization has more than suf- 
ficient assets to meet its estimated future liabilities. 





Figure 1: WSIB Unfunded Liability, 
1990-2008 ($ billion) 


Source of data: Ontario Workplace Safety and Insurance Board 
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Conversely, a low ratio indicates that the organiza- 
tion currently has insufficient assets to meet its 
estimated future liabilities. As of December 31, 
2008, Ontario’s funding ratio stood at 53.5%. By 
comparison, the funding ratio of the other four 
large provincial boards included in our review 
ranged from 70% to 116% and averaged 102%. 

Figure 2, which breaks out the unfunded liability 
by its asset and liability components, shows that, 
since 2001, the WSIB’s liabilities (consisting mainly 
of workers’ claims for benefits) have increased at an 
average rate of 2.5%, which is close to the 2.7% aver- 
age growth rate in the fund’s assets over that same 
period. Because the WSIB’s liabilities are so much 
larger than the fund’s assets, the generally consist- 
ent average growth rate in both the liabilities and 
the fund’s assets means the liabilities are increasing 
much faster in size than are the fund’s assets, and 
therefore the unfunded liability has gotten larger. 
On a year-over-year basis, the other key variable has 
been that the yearly rates of return on the fund’s 
assets have varied significantly since 2001. 

Figure 3 shows that Ontario’s unfunded liability 
dwarfs those of the several other larger provinces 
we compared it to. 


HISTORICAL PERSPECTIVE AND 
PROJECTIONS 


In 1984, the WSIB adopted what it called a “full- 
funding” strategy to retire the unfunded liability 
over a 30-year period. The full-funding strategy 
aimed to have the WSIB’s assets match its liabilities 
by 2014. From 1984 onward, achieving the goal of 
eliminating the unfunded liability by 2014 was the 
primary objective of the WSIB’s funding approach. 
By 1994, the unfunded liability stood at 
$11.4 billion. In response to the growing unfunded 
liability, the government reduced benefits in 1995. 
In 1996, the government of the day undertook a 
comprehensive review of the system. That review 
determined that legislative intervention was neces- 
sary because the existence of such a large unfunded 
liability threatened the viability of Ontario’s system. 


Figure 2: WSIB Liabilities, Assets, and Unfunded 
Liability, 1996-2008 ($ billion) 


Source of data: Ontario Workplace Safety and Insurance Board 
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Figure 3: Funded Position and Unfunded Liability for 
Major Provincial Boards in Canada, 2004-2008 
($ billion) 


Source of data: Various Provincial Workplace Safety and Insurance Boards 
and Workers’ Compensation Boards 
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The government believed that the unfunded liabil- 
ity not only put at risk the system’s ability “to pro- 
vide fair and secure compensation to injured [and 
ill] workers both now and in the future” but also 
had resulted in premium rates that were among 
the highest in North America—a situation that 
adversely affected Ontario’s competitive position. 
The 1996 review of Ontario’s workplace safety 
and insurance system led to the passage of the new 
Workplace Safety and Insurance Act, 1997 (Act). 
The Act made fundamental reforms to Ontario’s 
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workplace safety and insurance system by refocus- 
ing the system on the need for preventing injuries 
and encouraging early return to work. The Act con- 
tained new provisions that were intended to restore 
the system’s financial viability by reducing benefits 
to a level consistent with those in other provinces 
and by tightening eligibility requirements for work- 
ers seeking compensation. These legislative reforms 
resulted in a net reduction of $1.8 billion in future 
benefit costs and expected benefit liabilities. 

Subsequent to the passage of the Act, the WSIB 

has consistently maintained its commitment to 
deal with the size and growth of the unfunded 
liability and have the plan fully funded by 2014. For 
instance: 

e The 2005 funding framework reconfirmed 
the WSIB’s commitment to avoid passing the 
unfunded liability on to future generation of 
employers. 

e In 2006, the WSIB released its 2006-10 five- 
year strategic plan entitled The Road Ahead. 
This plan noted that, in the interests of finan- 
cial sustainability, the WSIB was “building 
a strong financial framework to address the 
elimination of the unfunded liability.” This 
five-year strategic plan was updated in Sep- 
tember 2007 as The Road to Zero, which covers 
the period 2008-12. The document notes that 
“the Board [remains]committed to a planned 
and disciplined approach to eliminating the 
Unfunded Liability and achieving full funding 
by 2014.” 

e The 2008 funding framework updated the 
2005 framework and formalized it for the 
next three years. In detailing how the WSIB 
would meet its full-funding target by 2014, 
the 2008 funding framework relied heavily on 
the targets set in The Road to Zero with respect 
to the WSIB’s commitment to eliminating all 
workplace injuries, illnesses, and fatalities 
and to reducing the amount of time it takes 
for injured or ill workers to recover and return 
safely to work. 


In our opinion, the WSIB’s publicly stated goal 
of achieving full funding status of the plan by 2014 
in the 2008 funding framework was ambitious. This 
expectation was premised on having all employ- 
ers and their workers meet all the objectives and 
targets set out in The Road to Zero. Specifically, 
the Board would have to reduce costs, increase 
premium revenues, and/or record investment gains 
that, when combined, would produce an average 
surplus of approximately $1 billion in each of the 
remaining eight years in order to eliminate by 2014 
the $8.1 billion unfunded liability that existed at 
December 31, 2007. However, given the $11.5 bil- 
lion unfunded liability reported at December 31, 
2008, the WSIB’s operating results would have to 
go from annual operating deficits averaging over 
$900 million in recent years to operating surpluses 
averaging $1.6 billion in each of the remaining 
seven years in order to eliminate the unfunded 
liability by 2014, as shown in Figure 4. 

The Board of Directors was advised, in presenta- 
tions on the 2009 corporate budget in November 
2008 and in a preliminary 2009 funding outlook 
presentation in December 2008, that the 2014 
target was no longer achievable. The Board was 
advised that the impact of the significant downturn 
in the global financial markets was the key driver 
for the re-assessment of the 2014 target. 





Figure 4: WSIB Unfunded Liability 1995-2008 and 
Future Projections (Annual Decline Required to 
Achieve Full Funding by 2014) ($ billion) 


Source of data: Office of the Auditor General of Ontario 
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In February 2009, largely as a result of the 
impact of the global financial crisis on the value of 
the WSIB’s investments, the WSIB prepared new 
estimates of the unfunded liability based on three 
scenarios: 

@ Base Case, 2009 Preliminary Funding 
Estimate—Under this scenario, the 2014 
unfunded liability would be $6.7 billion and 
its elimination was projected to occur in 2018. 

e Plausible Adverse Scenario—Factoring eco- 
nomic uncertainties and cost pressures into 
the Base Case, this scenario estimated the 
2014 unfunded liability to be $11.4 billion and 
projected its elimination in 2021. 

e Plausible Adverse with Indexation Scenario 
Base Case—This scenario factored the eco- 
nomic uncertainties and cost pressures into 
the Base Case, but also assumed continued 
full indexation for certain workers’ benefits, 
as required under Bill 187, which was an act 
respecting Budget measures and interim 
appropriations. It estimated the unfunded lia- 
bility in 2014 to be $14.1 billion and projected 
it would be eliminated no earlier than 2022. 


SHOULD WE BE CONCERNED ABOUT 
THE UNFUNDED LIABILITY? 


The risk that a large unfunded liability poses to the 
system’s financial viability was recognized over a 
decade ago. In the 1996 review that led to the legis- 
lative changes contained in Bill 99, it was noted 
that the problems associated with the unfunded 
liability could eventually come to a head when the 
WSIB was forced to liquidate investments in order 
to fund current operations and benefit payments. 
The 1996 review noted that between 1985 and 
1995, the WSIB had “transferred some $1.65 billion 
from the investment portfolio to general operations 
to pay for benefit payments.” A 1998 Statistics 
Canada research paper, Government Finances and 
Generational Equity, noted that “[i]t is compounded 
returns on these assets that the [WSIB] relies upon 
in order to pay future benefits. Removing the base 


of assets upon which investment revenues must be 
earned presages the descent into a vortex that will 
require strong action to reverse.” 

Withdrawals from the Investment Fund are still 
occurring: in the past seven years, the WSIB has 
transferred approximately $3.4 billion from invest- 
ments in order to fund payments in workers’ bene- 
fits. In addition, the WSIB had to sell $550 million 
more in investments than it has purchased in order 
to cover cash shortfalls from operations. The ser- 
iousness of this practice was conveyed in 2003 by a 
member of the Board of Directors who, as noted in 
the Board of Directors’ meeting minutes, indicated 
that there was a strong sentiment among members 
of the Investment Committee that the WSIB should 
avoid drawing funds from the Investment Fund and 
that one member of the committee indicated he or 
she might resign as a result of this issue. 

We acknowledge the argument that because 
the worker’s compensation system is a perpetu- 
ally ongoing operation, the unfunded liability is 
meaningless. From this point of view, this liability 
is merely an amount that will become due only in 
the highly unlikely event that the WSIB was to wind 
down its operations today. For example, the WSIB 
told us that, from an injured worker’s perspective, 
there are currently sufficient funds to pay benefits 
for up to three years even if ongoing premium rev- 
enue was eliminated completely and the WSIB was 
wound down. 

We do not agree with this argument and are 
concerned that the trend of selling off the WSIB’s 
investments to fund current operations and benefit 
payments is not financially sustainable. 

The more pertinent issue is whether a large 
unfunded liability poses significant risk to the 
financial viability of the system. To assess this con- 
cern, one needs to examine the major drivers that 
increase (or decrease) the unfunded liability—that 
is, the factors that drive either decreases in assets or 
increases in liabilities. 

Of the several drivers that either decrease assets 
or increase liabilities, the most notable for 2007 and 
2008 were actual investment returns in comparison 
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to the 7% expected return; increased costs resulting 
from legislated changes that increased benefits lia- 
bilities; and the cost to carry the unfunded liability 
itself (at 7% per annum). Over the last two years, 
$4.2 billion has been lost owing to investment 
losses; almost $900 million owing to additional 
injured-worker benefits; $1.3 billion owing to 
changes in actuarial estimates and assumptions; 
and $1 billion to carry the liability. Conversely, over 
these two years, only $2 billion has been collected 
from employers to offset this, with the end result 
that the unfunded liability has increased by almost 
$6 billion. 

Although the WSIB has suffered significant 
investment losses for 2007 and 2008, it does not 
expect these to continue in the long term. On the 
basis of average long-term returns in the equities 
markets, this is not an unreasonable assumption. 

If this turns out to be the case, the risk to financial 
viability arising from the size of the unfunded 
liability will therefore rest primarily with the other 
two components: whether premium rates are suffi- 
cient to cover current benefit payments and the cost 
to carry the unfunded liability and pay it down over 
a reasonable period of time; and whether legislated 
changes add additional unfunded costs to the sys- 
tem. For instance, under Bill 187, partially disabled 
workers were granted a 2.5% increase in each of 
2007, 2008, and 2009. There is added pressure 
from certain stakeholders to restore full inflation 
protection for 2010 and beyond. 


ELIMINATING THE UNFUNDED LIABILITY 


The WSIB has only three levers at its disposal to 
reduce the unfunded liability: 

@ increasing premium revenues; 

e reducing benefit costs (by reducing the num- 
ber and duration of benefit claims and health- 
care costs and/or by reducing or eliminating 
benefits); and 

e increasing investment income. 

This section discusses each of these tools and 

makes reference to some of the practices we found 


in the other provincial jurisdictions we visited. But 
it is important first to supply some perspective on 
some of the pressures and challenges the WSIB 
faces in its operating and political environment. 


Balancing Stakeholder Interests 


The Road to Zero communicates the WSIB’s five-year 
(2008-12) corporate strategy, which supports the 
WSIB’s vision of eliminating all workplace fatalities, 
injuries, and illnesses. The WSIB advised us that it 
uses an integrated approach to workplace health 
and safety that promotes collaboration among sys- 
tem partners and stakeholders through initiatives 
such as a prevention partnership model, rewarding 
superior workplace behaviour, and facilitating a 
cultural shift through social marketing campaigns. 
The WSIB indicated that key initiatives in The Road 
to Zero include reducing injured workers’ time off 
work, bringing a more integrated approach to case 
management, facilitating early and safe return to 
work, purchasing cost-effective and evidence-based 
health-care services, managing occupational- 
disease services more effectively, and a new service 
delivery model, which is expected to help reduce 
benefit costs and optimize premium revenue while 
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improving overall service. 
As The Road to Zero communicates, the WSIB’s 
efforts to improve its funding position have focused 





primarily on reducing the number of new claims 
and reducing claims duration. Increasing premiums 
or reducing benefits has proven difficult—many 
years of experience have shown the inherent 
political, social, and economic sensitivity of 
implementing changes to either. But the last two 
years have also demonstrated how unpredictable 
and inconsistent the results can be when the third 
option of overly relying on increased investment 
returns is chosen. 

Governments and workplace safety/workers’ 
compensation organizations face significant pres- 
sures in trying to manage the system. The problem 
faced by the government and the WSIB when trying 
to eliminate the unfunded liability is that there is 
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little agreement among labour groups and employ- 
ers as to whether the unfunded liability is even a 
problem, let alone how it should be addressed if it is 
a problem. The positions generally taken by labour 
groups and employers differ in ways that reflect a 
concern that they will be required to bear the entire 
cost of eliminating the unfunded liability. 

Labour is concerned that the unfunded liability 
will be eliminated by reducing benefits to workers 
and by restricting the eligibility of certain types of 
injuries and occupational diseases for compensa- 
tion. Many employers believe that the unfunded 
liability exists only because benefit levels are too 
high and because the scope of injuries and occu- 
pational diseases being compensated is too broad. 
In addition, they believe that increasing premium 
rates to eliminate the unfunded liability would hurt 
both their competitive position and job creation. 
Employers therefore tend to believe that premiums 
should not be increased and that, instead, benefit 
reductions represent the only acceptable approach 
to reducing the unfunded liability. Both the govern- 
ment and the WSIB must balance the concerns of 
labour and employer stakeholders. This balancing 
act between changing employer premiums and/or 
changing worker benefits is where the WSIB is most 
susceptible to the influence of the government of 
the day. 


Authority of the Board to Address the 
Unfunded Liability 


The Ontario Financial Review Commission’s 
(OFRC’s) 1995 report on the province’s accounting, 
reporting, and financial-management practices 
commented on the government’s apparent influ- 
ence over benefits, premiums, and coverage, which 
the OFRC believed undermined the WSIB’s ability 
to govern itself in an accountable fashion. The 
OFRC stated that “while the government has the 
responsibility for setting the Board’s mandate, the 
Board must have the sole power to carry it out.” Our 
observation is that the OFRC’s comments continue 
to be relevant. 


For example, in 2002, one WSIB director com- 
mented (as recorded in the Board of Directors’ 
meeting minutes) that “some government repre- 
sentatives are of the view that the Board should 
reduce premium rates [for 2003, which was a 
provincial general election year] below the level 
required to meet claims costs.” 

We noted that the funding models of the two 
other provincial boards we visited require that their 
plans be fully funded—and that the officials at one 
board even consider this full-funding requirement 
to be enshrined in their governing legislation. 
Although Ontario’s legislation does not require a 
fully funded plan, the government and the WSIB 
may wish to consider whether there is lack of clarity 
around the role of the WSIB and of the govern- 
ment in ensuring that the system is managed ina 
financially accountable manner and that the plan 
remains financially viable. For example, we noted 
that one of the guiding principles in the Memoran- 
dum of Understanding (MOU) between the WSIB 
and the government is that the “Minister and the 
Board are committed to a workplace safety and 
insurance system predicated on sound insurance 
and business principles, including...a commitment 
to achieving and maintaining a financially sustain- 
able workplace insurance plan.” Missing from the 
MOU’s described elements of sound insurance 
and business principles is a specific statement 
that the system must be based on sound financial 
management practices that will facilitate a fully 
funded plan. In the insurance business, sound 
financial management practices include ensuring 
the system’s financial stability and sustainability 
for the beneficiaries not only by making the plan 
fully funded but also by setting aside reserves to 
address any major financial shocks that may affect 
the system. 


Premium Revenues 


In 1996, the average premium rate was $3 per 
$100 of payroll—a decrease from the 1991 average 
premium of $3.20. Despite the WSIB’s apparent 
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authority to set premium rates, the government’s 
May 7, 1996, Budget Speech announced a planned 
5% reduction in the average premium rate (to 
$2.85) effective January 1, 1997. Since then, 


notwithstanding the unfunded liability’s upward | Rate | 


trend, the average premium rate has been reduced 1991 


Figure 5: WSIB Average Premium Rates, 1991-2009 
($ per $100 of Insurable Payroll) 


Source of data: Ontario Workplace Safety and Insurance Board 
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Figure 6: WSIB, Comparison of Benefit Expenses, 
Benefit Costs Paid, and Premium Revenues, 
1998-2008 ($ billion) 


Source of data: Ontario Workplace Safety and Insurance Board 


annual expenses. In fact, they just barely cover 
the actual cash amounts of benefits paid, with 
none of the surcharge actually going toward 
paying down the WSIB’s unfunded liability. 


Figure 6 illustrates this point. : (iy premium revenues for cost of current year claims 


[J unfunded liability premium charge 
=@- benefit expenses (accrual basis) 
=e benefit costs paid (cash basis) 





e Premium revenues in Ontario cover a lower 






percentage of the WSIB’s expenses than pre- 





mium revenues cover in the four other large 
provincial boards we considered. As Figure 7 
shows, premium revenues from 2004 to 2008 3 
have covered on average only 70% of the 

WSIB’s total expenses. For each of the other 2 
provincial boards, premium revenues covered 
a higher percentage of total expenses over this 
same period—between 80% and 103%, with 
an average of 86%. 
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Figure 7: Major Provincial Boards, Total Gross 
Premium Revenues as a Percentage of Total Expenses, 
2003-2008 (%) 


Source of data: Ontario Workplace Safety and Insurance Board and Provincial 
Workers’ Compensation Boards 
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@ In reviewing the WSIB’s Statement of Oper- 
ations over the last 10 years, we noted that the 
WSIB has not reported an annual surplus since 
2001, and since that date, the annual deficits 
(that is, losses on operations) have ranged 
from a low of $142 million in 2006 to a high 
of $2.4 billion in 2008, with the annual deficit 
averaging over $900 million since 2002. 
Admittedly, investment losses in 2008 have 
been significant owing to the global economic 
downturn and have contributed over $3 bil- 
lion to the increase in the unfunded liability 
in 2008. However, the WSIB’s lack of success 
in eliminating the unfunded liability has been 
more directly the result of benefit expenses 
not being adequately funded by the premium- 
revenue and investment-revenue streams. 
More specifically, if previously recorded 
unrealized gains on the WSIB’s investments 
are netted against the investment losses in 
2008, the unfunded liability would likely still 
have been in excess of $7 billion. 


Premium Setting in Other Jurisdictions 
In each of the other two provincial jurisdictions we 
visited, there was no ambiguity regarding how they 
establish their premium rates: each sets rates based 
on the principle that its plan must be fully funded at 
all times. For example: 

@ One provincial board’s Act imposes a statu- 
tory obligation on the board to ensure that its 
plan is fully funded: “The Board must ensure 
that there is sufficient money available in the 
Accident Fund for the payment of present 
compensation and future compensation 
as estimated by the Board’s actuary.” The 
board’s rate-setting policy rests on a phil- 
osophy that includes eliminating the use of 
investment returns to subsidize average pre- 
mium rates. According to the board’s funding 
policy, its plan is considered to be sufficiently 
funded only when the plan’s total assets equal 
or exceed its total liabilities, and its plan is 
not considered to be fully funded unless its 
funded ratio is within the target range of 
114% to 128%. If the funded ratio falls below 
that target range, that provincial board’s 
funding policy requires that special levies be 
included in the premium rates to recover the 
shortfall. 

@ Officials from the second provincial board indi- 
cated that, although its governing legislation 
includes no specific provision requiring full 
funding, they have adopted an external bench- 
mark for determining a suitable funding level 
for their plan. The benchmark chosen—that 
used in the insurance sector as legally required 
by that sector’s regulator (the Office of the 
Superintendent of Financial Institutions) — 
requires insurers to hold prescribed levels 
of capital to help mitigate against possible 
adverse circumstances and to be able to con- 
tinue operating. The minimum capital reserve 
requirements therefore depend on the risk 
inherent in the assets (for example, equities 
require relatively more capital reserves than 
bonds) and in the liabilities (for example, 


Unfunded Liability of the Workplace Safety and Insurance Board 327 


liability estimates that have greater uncer- 
tainty require more capital reserves). The 
provincial board officials told us that adopting 
these capital adequacy guidelines has had the 
additional benefit of providing clarity to their 
stakeholders, with the result that stakeholders 
had no expectation of receiving large-scale 
refunds of the surplus that had built up during 
the four years before the 2008 declines in the 
investment markets. Stakeholders understood 
that capital reserves could be used to dampen 
the impact on employer premiums during dif- 
ficult economic periods. 

In reviewing the process for setting premium 
rates in the two provincial jurisdictions we visited, 
we also noted that the underlying funding model 
obliges them to ensure that annual premiums 
charged to employers during a given year cover 
the cost of all new claims, the future costs of these 
claims, and related operating costs incurred during 
the year so that future employers do not bear the 
costs of prior-year worker injuries: 

e Officials from one provincial board noted 
that they ensure that premium revenues meet 
or exceed all claim costs plus all overhead 
incurred during the year. They set a rate for 
each rate group based on that rate group’s 
injury experience. Rates within that rate 
group are then adjusted for safety association 
levies, claims administration, appeals com- 
mission, medical panels, and occupational 
disease reserve funding. They noted that char- 
ging employers with the full cost of workplace 
injuries through premium rates “is imperative 
to securing workers’ benefits for the lifetime 
of the claim and not passing costs on to future 
employers.” 

@ Officials from the second provincial board 
noted that they ensure that current premium 
rates reflect the system’s current costs: their 
target rate is the rate required to ensure 
that the system remains fully funded. They 
indicated they ensure that premium levels 
and corresponding benefit costs are evalu- 





ated by industry. In summary, they stated 
that their funding strategy’s overall objective 
is “full funding at the rate group level (and 
consequently for the system as a whole) with 
additional appropriated reserves” provided 
“as deemed necessary.” 

e The officials from the second provincial board 
also noted that their organization has had a 
longstanding “unique practice of maintaining 
separate segregated industry rate group 
results and funding levels that are transparent 
to employers.” That is, their board tracks each 
rate group’s results by business activity (for 
example, premium revenues, claims costs, 
and unfunded liability/funded asset position) 
and by injury risk (for example, number of 
claims, accident types, nature of illnesses, 
injury rates, and claims duration). In their 
view, this “open-book policy” of tracking and 
reporting the performance of a particular 
industry rate group or industry sector on the 
types of injuries and illnesses taking place 
in that industry rate group, as well as on the 
costs of the workplace injuries and illnesses 
of that sector, has had a very positive effect of 
“garnering employer support for [premium] 
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rate increases when necessary in order to 





recover from deficits or to lower accident rates 
and injury costs.” They commented that their 
funding policy of requiring premium assess- 
ment rates to cover the costs incurred during 
the year, supported by this segregated indus- 
try rate group data, has the additional benefit 
of making it very difficult for the government 
of this province to influence premium rates. 


Ontario’s Approach to Setting Premiums 
Setting premium rates that reflect the “full target 
level”—that is, the level required to cover the 
anticipated full costs of new claims, including 
administrative charges and unfunded liability 
charges—ensures that firms and industries pay for 
their costs relating to injuries or deaths incurred in 
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their workplaces. The WSIB has stated that it fol- 
lows a similar approach. As the 1996 review noted, 
Ontario employers are classified in nine industry 
classes, which are divided into rate groups. 

The WSIB advised us that in 2008 there were 
154 employer rate groups based on similar business 
activities and workplace health and safety risks. 
Premium rates are set each year on the basis of pro- 
jecting a rate group’s health and safety performance 
from the previous five years to the premium-rate 
year. Premium rates comprise four components: 

® costs of new claims; 


a charge to repay the unfunded liability; 


gain or loss component; and 


administration expenses covering the 
expected costs to run the WSIB and adminis- 
ter the Workplace Safety and Insurance Act, the 
costs of the Ministry of Labour to administer 
the Occupational Health and Safety Act, fund- 
ing for the health and safety associations, 
research, and the costs of the Offices of the 
Worker and Employer Advisers. 

Under this methodology, premium rates can go 
up or down depending upon a rate group’s work- 
place health and safety performance. When the 
average premium rate remains unchanged, about 
one-third of the rate groups have a rate increase, 
one-third stay the same, and one-third have a rate 
decrease. 

In using this approach, the WSIB faces a chal- 
lenge when unanticipated events, such as the 
recent economic downturn, result in either higher 
costs or lower revenues than planned. Ideally, such 
unanticipated shortfalls would be recovered by 
increasing subsequent years’ premiums. However, 
as Figure 5 illustrates, average premium rates are 
12.7% lower now than 10 years ago. Notwithstand- 
ing this, we recognize that increasing premium 
rates, and thereby increasing the cost of doing 
business in this province, would not be a popular 
option for either the WSIB or the government in the 
midst of a severe economic downturn. This problem 
is compounded by the fact that Ontario’s average 
premium rate is among the highest in Canada (see 


Figure 8). Ontario will eventually need to increase 
its premium rates if it hopes to make any progress 
toward eliminating its unfunded liability—unless 
downward revisions are made to the current 
benefits structure or investment returns recover 
dramatically. 

Premium revenues are also affected by the 
number of workers covered by the workers’ com- 
pensation system. As the 1996 review pointed out, 
coverage of Ontario’s workforce was significantly 
lower than in many other provinces. The 1996 
review suggested that covering more workers 
might create additional revenues for the WSIB. 
Ontario’s coverage rate remains among the lowest 
in the country. 

In April 2009, the WSIB analyzed the additional 
revenues it would earn if coverage were expanded 
to include the entire Ontario workforce. Estimated 
additional revenues would be approximately 
$280 million. Included in this is $72 million that 
will be earned beginning in 2012 resulting from Bill 
119, The Workplace Safety Amendment Act,which 
will make insurance coverage mandatory for a 
number of construction workers who had not previ- 
ously been covered. The incremental revenues from 
expanding coverage would not come close to solv- 
ing the unfunded liability problem. But one advan- 
tage of increasing coverage to all workers is that it 
would stabilize and improve the WSIB’s financial 
position as the economy restructures. Many indus- 
tries that are not currently covered are in sectors, 
such as financial institutions, that are experiencing 
high employment growth, whereas, for example, 
26% of the WSIB’s premium revenues in 2008 came 
from the currently ailing automotive industry. 

And finally, in a worst-case scenario, the Act 
does allow the government to provide loans to the 
WSIB if the WSIB is ever unable to pay workers the 
benefits that they are entitled to. We are not aware 
of this having occurred previously in any workers’ 
compensation system in Canada. In Ontario, the 
government has provided financial assistance in the 
form of interest-free loans to the Pension Benefits 
Guarantee Fund (PBGF). The PBGF’s mandate is to 
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Figure 8: Comparison of 2009 Average Premium Rates, 2009 ($ per $100 of Insurable Payroll) 


Source of data: Association of Workers’ Compensation Boards of Canada 
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that have private-sector pension plans and there- 
fore not included in the province’s financial results. 
The province had to step in and provide the PBGF 
with the interest-free loans because the PBGF’s 
potential liabilities far exceeded the assets available 
to pay those liabilities. 


Benefit Costs 


In accordance with the Act, the WSIB provides 
benefits for loss of earnings, benefits for permanent 
injuries, payments for health care, assistance to 
facilitate return to work, and survivor benefits in 
the case of work-related fatalities. Collectively, 
these payments are termed “benefit costs”. 
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From 1999 to 2008, the WSIB’s benefit costs 
increased by about 7% annually—almost doub- 
ling—from $2.17 billion as of December 31, 1998, 
to $4.26 billion as of December 31, 2008. This is 
generally in line with the growth rates of benefit 
costs of the other provincial boards we compared 
Ontario against. 

The WSIB has identified increasing claims 
duration, which is the average number of days an 


injured worker is on benefits, as the key contributor 
to the increase in benefit costs since 1998. 


Legislative Changes Affecting Benefits Costs 
The 1997 legislative changes under Bill 99 gener- 
ally reduced worker benefits, but the 2007 changes 
increased them. Both changes undoubtedly 
reflected the policy agenda of the government of 
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the day. Before Bill 99, the legislative framework 
was highly structured and prescriptive regarding 
both the process and the timelines for handling 
claims. This system had numerous mandatory 
reviews or checkpoints, extensive WSIB interven- 
tions between worker and employer, and a strong 
focus on vocational rehabilitation. Bill 99 changed 
these processes significantly: its legislative reforms 
were based on a “self-reliance model”, which 
follows the philosophy that workplace parties— 
employers and workers—are in the best position to 
make practical decisions about the management of 
workplace injuries and that the workplace parties 
should therefore co-operate. For the WSIB, these 
reforms resulted in fewer and less-prescriptive 
policies and interventions, eliminated vocational 
rehabilitation, and refocused the WSIB’s role from 
one of direct intervention to one of monitoring the 
workplace parties. The WSIB indicated that this 
may have had the unintended effect of increasing 
claims duration. The legislation also required the 
WSIB to increase its activities aimed at preventing 
workplace injuries. Bill 99 also reduced the infla- 
tion protection provided to partially disabled 
workers. 

The 2007 legislated changes, announced in 
the 2007 Ontario Budget, introduced a tempor- 
ary indexing factor that increased some workers’ 
benefits for three years. The amendments became 
law when the Legislature passed the 2007 Budget 
Bill on May 17, 2007, and affected benefit payments 
beginning on July 1, 2007. The amendments, 
among other things, temporarily increased benefits 
for a number of workers by 2.5% on July 1, 2007, 
2.5% on January 1, 2008, and a further 2.5% on 
January 1, 2009. The government decision to 
increase benefits added $750 million to expenses 
and to the unfunded liability reported by the WSIB 
in its December 31, 2007, financial statements. 

If the government were to introduce further 
benefit increases after January 1, 2009, similar to 
those implemented in the previous three years, the 
WSIB estimates that such changes would increase 
its expenses by $1.6 billion and add $1.6 billion 


to the unfunded liability in 2010 (because, under 
generally accepted actuarial standards, the WSIB’s 
actuary would likely need to assume that this 
indexing rate had at that point become perma- 
nently built into the system). 

Our Office is not questioning the government’s 
policy decision to increase workers’ benefits—the 
government has the sole responsibility for setting 
benefits and coverage through legislation. However, 
we do want to highlight how a government’s deci- 
sion to increase benefits can impair the WSIB’s abil- 
ity to address the unfunded liability. The problems 
inherent in increasing benefits without adequate 
financial provision were highlighted in the 1996 
review that led to the legislative changes contained 
in Bill 99: 


However, the costs of [the] improvements 
[introduced in 1989 and 1994] were 

not balanced by measures to guarantee 
adequate reserves to meet current and 
future obligations. Understandably, 
expansion and enrichments in the name 
of improved equity have proved popular. 
However, governments in the past have 
chosen not to address the critical but 
difficult problem of how to finance these 
benefit changes. 


In 2007, concerns about the rising benefit costs 
prompted the WSIB to engage the Institute for 
Work and Health (Institute) to examine its claims- 
duration data. That study identified three primary 
drivers of increased claims duration: 

e Legislation: There was a high correlation 
between longer duration outcomes and the 
changes made by Bill 99 to the legislation 
and the Board’s service delivery model (for 
example, Bill 99’s elimination of mandatory 
reviews and checkpoints, along with the 
Board’s vocational rehabilitation programs). 

@ Health Care: There was a high correlation 
between duration and health-care services 
being received, especially with prescriptions 
for narcotic medication. 
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e Workplace Behaviour: There was a high cor- 
relation between duration and changes in 
workplace behaviour that are affected primar- 
ily by the WSIB’s workplace-safety employer- 
incentive programs. 

The study concluded that only systemic change 
would make it possible for the WSIB to reverse the 
longstanding upward trend in claims duration. 

The Institute’s study showed that before Bill 99, 
claims duration was decreasing. Although Bill 99 
was intended to decrease claims duration, it 
actually contributed to some significant unantici- 
pated increases in claims duration. For example, 
the percentage of locked-in claims relative to long- 
term claims almost doubled from 1997 to 2001. 

The Institute also examined why the self- 
reliance model had not achieved the intended 
result of decreasing claims duration, concluding 
that the primary reason was the ineffectiveness of 
the WSIB’s return-to-work incentive programs. For 
the self-reliance model to succeed, the WSIB would 
need to have incentive programs that motivate 
employers to improve both prevention and return- 
to-work practices. 

From an operational standpoint,the WSIB 
advised us that to reduce claims duration, it is 
introducing a new service delivery model that 
includes changes in how the WSIB manages claims 
both before and after accepting them. For example, 
changes are being made to the roles and function 
of the WSIB’s claims adjudicator, and interven- 
tions will occur earlier than has been the practice 
in recent years. This revamped process will be 
supported by a new service delivery technology 
system that will provide the adjudicator with better 
information with which to make decisions. This 
new technology will be phased in, with complete 
implementation expected by mid-2010. 

Although it is hoped that these initiatives 
will help reduce claims duration, fundamental 
legislative changes may also be needed before any 
significant progress can be made in reducing claims 
duration. 


The following sections discuss the previously 
mentioned health-care and workplace behaviour 
issues in more detail. 


Health-care Costs 
Health-care costs are those medical costs incurred 
by the injured worker that are paid by the WSIB. 
Health-care costs paid by the WSIB on behalf of 
workers receiving benefits averaged 16% of total 
benefit costs over the 1999-2008 period. But in 
that same period, these health-care costs more 
than doubled—rising from $238 million in 1998 to 
$619 million in 2008. 

One of the primary drivers of increased health- 
care costs is the increased number of narcotic pre- 
scriptions for analgesia (pain relief). The Institute 
for Work and Health’s study reported that in the 
nine-year period from 1999 to 2007 the number of 
such prescriptions included in the WSIB’s health- 
care costs more than doubled (see Figure 9). Pre- 
scription costs grew from 20% to over 44% of the 
total health-care costs paid annually by the WSIB. 

The increased use of these drugs concerns the 
WSIB from two standpoints: the drugs’ direct costs 
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Figure 9: WSIB, Number of Narcotic Analgesic 
Prescriptions Included in Health-care Costs, 
1999-2007 


Source of data: Ontario Workplace Safety and Insurance Board 
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and the drugs’ high potential for creating depend- 
ency. The Institute report noted that dependency 
on such drugs can often delay an individual’s return 
to work and can increase the chances of an injury 
occurring when the individual does return to work. 
Studies have also shown that the addiction nega- 
tively affects the efforts of other return-to-work 
programs. Experience has shown that it is import- 
ant to get the worker back to his or her workplace 
as soon as safely possible, because the longer the 
worker is away from the job the more difficult it 
is to return to work. Thus, any factor that tends 
to delay the return to work will increase claims 
duration (and therefore benefit costs) for both the 
short term and the longer term. 

The WSIB, however, has minimal involvement 
in determining the need for these drugs, which 
is typically decided by the patient’s doctor. This 
limitation restricts the WSIB’s influence over the 
costs involved. Nevertheless, the WSIB is in the 
process of rolling out a narcotic drug program. This 
program includes the establishment of a narcotic 
drug advisory committee, and the WSIB will work 
with the Ontario Medical Association on ways to 
better inform treating physicians about the risks 
associated with these drugs. The WSIB hopes these 
collaborative efforts will succeed in reducing the 
use of such drugs. But if these efforts fail, the WSIB 
believes the only option would then be to consider 
the introduction of standards governing the situa- 
tions under which it will and will not pay for pre- 
scribed analgesic narcotics. 


Workplace Behaviour/Incentive Programs 
The WSIB has a number of incentive programs 
of the type referred to as “experience-rating pro- 
grams.” Such programs reward employers (typically 
through rebates) for results that reflect good prac- 
tices and penalize (typically through surcharges) 
employers for poor results. For example, employers 
whose injury rates are lower than the average for 
their rate group are rewarded; those whose injury 
rates are higher than their rate group’s average are 


penalized. The Institute pointed out that if claims 
duration in general is increasing, rebates should 
decrease and/or surcharges should increase cor- 
respondingly. But the study noted that the opposite 
was occurring: employers were still being rewarded 
even as their injured-worker claims duration was 
increasing. 

The WSIB recognized this issue and engaged an 
external consulting firm in June 2008 to review the 
programs. The report’s key findings included the 
need for “better integration of the Board’s experi- 
ence rating programs with its legislative obliga- 
tions,” and “improved fairness and equity so that 
the programs enhance the focus on real improve- 
ments in prevention and return-to-work outcomes.” 

In response to the consultant’s reeommenda- 
tions, the WSIB began obtaining input from 
stakeholders during 2009, with a view to revising 
the experience-rating programs to address those 
concerns. 


Reducing New Claims 
Reducing the amount of new claims entering the 
system is also critical to controlling the growth of 
benefit costs. Both the WSIB and the other major 
provincial boards focus on initiatives aimed at pre- 
venting work-related injuries and illnesses. In our 
discussions with the WSIB and the two other prov- 
incial boards we visited about lost-time injury rates, 
the consistent message we heard was that their 
injury-prevention initiatives have improved aware- 
ness of health and safety issues among employers 
and workers and have resulted in lower lost-time 
injury rates than those reported 10 years ago. But 
the other provincial boards also commented that, 
despite such efforts, their levels of new claims for 
lost-time injuries have remained relatively flat over 
the last few years. 

In its 2008 funding framework, the WSIB set a 
goal of attaining a 7% reduction in new claims each 
year from 2008 through 2012, which is consistent 
with the 35% target for reduction of lost-time injury 
rates set out in the The Road to Zero strategy for that 
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same period. The target assumes that the WSIB’s 
prevention efforts will result in significant reduc- 
tions. But the 2008 target was not met. According 
to the WSIB, the factors that prevented its achieve- 
ment were that year’s lower-than-anticipated 
employment—and therefore premium—growth 
and the worse-than-expected lost-time injury rates 
of two large industry sectors. 


Investment Income 


The WSIB relies on both premium revenues and 
investment income to fund its current and future 
obligations. The carrying value of the Insurance 
Fund’s investments as of December 31, 2008, was 
$11.1 billion. This amount represented a decrease 
of $2.6 billion from the December 31, 2007, balance 
of $13.7 billion—a decrease caused primarily by 
the significant decreases in global financial mar- 
kets. Investment results are reviewed by the WSIB’s 
Investment Committee, which includes as advisors 
external investment and financial experts. 

In most pension and insurance funds, invest- 
ment returns constitute an important source of 
revenue to fund operating expenses, pay future 
benefits, and increase the net assets held that are 
needed to pay for increases in expected future 
benefit payments. Therefore, having too few invest- 
ments relative to the WSIB’s liabilities and liquidat- 
ing investments to pay current operating expenses 
and benefit claims typically have a significant nega- 
tive impact on the size of the unfunded liability and 
fiscal sustainability of the WSIB. 

Our review of the WSIB’s investments during 
the past two years shows that they were diversified 
among three primary asset classes: public equities, 
59%; public fixed-income securities, 35%; and real 
estate, 6%. 

When looking at investment performance as it 
relates to the unfunded liability, it is important to 
look at investments’ long-term performance rather 
than their short-term performance, which can be 
very volatile. As Figure 10 shows, for 1994 through 


2008, the WSIB’s investments earned a 15-year 
average rate of return of 6.6%. 

Most of the WSIB’s long-term strategies for 
reducing the unfunded liability by 2014 have 
assumed a 7% average rate of return on its invest- 
ments. Given that actual returns have been less 
than 7%, investment performance has not con- 
tributed to reducing the unfunded liability, which 
is now one of the largest in the WSIB’s history. In 
an insurance operation such as the WSIB’s, where 
future benefits payments are expected to rise by 
7% annually, investments must earn more than 7% 
before any reduction in the unfunded liability can 
be realized from investment returns. In our view, 
this situation reinforces our observation that inad- 
equate premium revenues in comparison to benefit 
levels—rather than inadequate long-term invest- 
ment returns—are the main reason why benefit 
costs are not being fully funded. 

It appears to us that, given the pressures to 
keep premium rates low, expectations were placed 


Figure 10: WSIB, Average Rate of Return on 
Investments, 1994-2008 


Source of data: Ontario Workplace Safety and Insurance Board 
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on achieving very high investment returns as the 
vehicle for solving the unfunded liability issue. 
These pressures also may have contributed to the 
Board’s adoption of an investment policy of having a 
relatively high exposure to equities—in the hope of 
generating high returns—than the more conserva- 
tive investment policies pursued by the two other 
boards we visited. Figure 11 compares the WSIB’s 
investment mix with those of the two provincial 
boards we visited. The WSIB’s public-equity hold- 
ings have represented a much higher proportion of 
its total investments than has been the case with the 
other two organizations. Although having higher 
public-equity holdings can result in increased 
returns, it can also increase the risk, volatility, and 


losses in the WSIB’s investment returns compared to 
the other organizations. 

In 2006, at 16.2%, the WSIB had the best invest- 
ment returns of any provincial board in Canada. 
But as Figure 12 shows, when the economy started 
to slow in 2007, the WSIB’s investment returns 
dropped to negative 1%, causing the WSIB to rank 
10th in Canada. In 2008, all provincial boards suf- 
fered significant losses. 

It appears to us that one consequence of having 
good investment returns in any one year seems to 
have been a strong temptation for the government 
to assume that those gains can be used to finance 
enhancements to workers’ benefits. We noted that 
the Bill 187 legislative reforms—which added 





Figure 11: Comparison of Investment Mix in Provincial Jurisdictions Reviewed (%) 
Source of data: Ontario Workplace Safety and Insurance Board and Provincial Workers’ Compensation Boards 
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Figure 12: One-year Rates of Return on Investments, All Provincial Boards, 2004-2008 


Source of data: Association of Workers’ Compensation Boards of Canada, 2004-2007, and 
Ontario Workplace Safety and Insurance Board and Provincial Worker's Compensation Boards, 2008 
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2004 | 8.5 8 7.6 
2005 10.5 7 9.5 
2006 16.2 1 11.6 
2007 (0.7) 10 (0.7) 
2008 (15.5) 4 (26.9) 


$750 million for 2007 by introducing a temporary 
increase in the indexation of workers’ benefits for 
2007 through 2009—followed shortly after the 
impressive 16.2% investment returns experienced 
in 2006. 

To reduce the effects of market volatility and to 
better match its investments to its long-term fund- 
ing obligations, the WSIB revised its investment 
policies as of January 1, 2009. Its new Statement 
of Investment Policies and Procedures continues to 
have a long-term investment return objective of 7%, 
but proposes a significant change in the asset mix. 
The new asset mix strategy emphasizes reducing 
risk by the public equity component to 25% from its 
current 59% level and includes private equity, infra- 
structure, and additional real estate investments. 
We were advised by the WSIB that this approach 
is supported by the Ministry of Finance and is 
expected to lower volatility while still achieving the 
target investment return of 7%. 


TOMORROW’S EMPLOYERS PAYING FOR 
COSTS OF TODAY 


Section 96 (2) of the Workplace Safety and Insur- 
ance Act, 1997 states: 


Rate of Return on 
Investments Range 


Avg. Rate 


of Return 
(%) 





12.8 9.8 
15.7 11.9 
16.2 14.1 
6.5 ae 
(8.2) (16.5) 


The Board has a duty to maintain the 
insurance fund so as not to unduly or 
unfairly burden any claims of Schedule 1 
employers in future years with payments 
under the insurance plan in respect of 
accidents of previous years. 


Clearly, the very existence of the unfunded 
liability demonstrates that, over the years, the 
province’s employers have not fully funded the 
costs of injuries and occupational diseases, so these 
liabilities will need to be funded by future employ- 
ers.Thus, employers in currently declining industry 
sectors have transferred workplace-safety financial 
obligations to other current and future generations 
of employers. 

Given the government’s legislated role in deter- 
mining benefit levels and employees to be covered 
by the system, addressing this section of the Act is 
not entirely within the Board’s purview. However, 
we urge both the government and the WSIB to keep 
the intent of this section of the legislation in mind 


when making future premium and benefit decisions. 
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It is our practice to make specific recommendations 
in our value-for-money (VFM) audit reports and 
ask ministries, agencies, and organizations in the 
broader public sector to provide a written response 
to each recommendation, which we include when 
we publish these audit reports in Chapter 3 of our 
Annual Report. Two years after we publish the rec- 
ommendations and related responses, we follow up 
on the status of actions taken by management with 
respect to our recommendations. 

Chapter 4 provides some background on the 
value-for-money audits reported on in Chapter 3 
of our 2007 Annual Report and describes the 
status of action that has been taken to address our 
recommendations since that time as reported by 
management. 


Follow-up on 2007 
Value-for-money Audits 


For over 90% of the recommendations we made 
in 2007, management has indicated that progress 
is being made toward implementing our recom- 
mendations, with substantial progress reported for 
over half. 

Our follow-up work consists primarily of inquir- 
ies and discussions with management and review of 
selected supporting documentation. In a few cases, 
the organization’s internal auditors also assisted 
with this work. This is not an audit, and accordingly, 
we cannot provide a high level of assurance that the 
corrective actions described have been implemented 
effectively. The corrective actions taken or planned 
will be more fully examined and reported on in 
future audits and may impact our assessment of 
when future audits should be considered. 







Section 





Background : 


The Archives of Ontario (Archives) has a broad 
mandate to oversee and manage recorded informa- 
tion in paper, electronic, and other forms created 
by ministries and most agencies, and to preserve 
recorded information of historical and permanent 
value and make it accessible to the public. 

Records that are not needed on an ongoing 
basis by a ministry or agency are transferred to and 
stored in warehouses operated by the government’s 
Information Storage and Retrieval unit (ISR), 
which has been the Archives’ responsibility since 
April 1, 2007. Storage periods for these semi-active 
records, which are set using a record-retention 
schedule authorized by the Archives, usually range 
from five to 100 years. After that, the records are 
to be either destroyed or transferred to the custody 
and ownership of the Archives. 

Ministries and most agencies must follow a 
government directive that sets out requirements for 
the storage of recorded information. The Archivist 
of Ontario ultimately has sole responsibility for 
approving the final preservation or disposal of all 
documents and records. - 


Eee Ministry of Government Services 


“RikE Archives of Ontario and 
Information Storage and 
Retrieval Services 


Follow-up on VFM Section 3.01, 2007 Annual Report 


Access to the Archives’ collections can be 
obtained through its reading rooms, the Internet, 
and public libraries via the interlibrary-loan 
microfilm program. Annually, customer inquiries 
and access requests for the archival and art collec- 
tions include 16,000 visits to the Archives’ reading 
rooms, 21,000 research requests, 7,000 microfilm 
loan requests, and approximately 70 million visits 
to the Archives’ website. 

In the 2008/09 fiscal year, the Archives’ 
expenditures were $18.8 million ($16.7 million in 
2006/07) and the Archives employed about 100 
staff. In April 2009, the Archives of Ontario com- 
pleted its move to a new head-office facility in north 
Toronto. 

In our 2007 Annual Report, we concluded that 
although the Archives had recently introduced a 
number of initiatives to upgrade its facilities and 
information systems, it did not yet have adequate 
systems and procedures to ensure that informa- 
tion of historical significance was being identified, 
stored, or archived safely and securely, and made 
readily accessible to users. The large growth and 
sheer volume of records destined for the Archives, 
both paper and electronic, exacerbate the challenge 
of identifying and cataloguing archival records of 
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historical value. Our more significant observations 
were as follows: 

@ The Archives did not have adequate systems 
and procedures for ensuring that the more 
than 10,000 record-retention schedules of 
government ministries and agencies, which 
are used to identify records with archival 
value, were complete and up-to-date. 

@ The Archives had no information on whether 
ministries and agencies were following 
required storage and archival policies. 

@ The Archives needed to be more proactive 
in working with ministries and agencies 
to reduce the significant growth in stored 
records. 

@ The Archives did not have a comprehensive 
strategy for dealing with the extensive elec- 
tronic documents that will need to be archived 
nor the technical expertise and capacity 
necessary to store and make them available to 
the public. 

® We found a number of weaknesses in inven- 
tory control practices, which may have 
resulted in significant losses, including thefts, 
over the years. 

@ Many archival records were not readily access- 
ible to the public because they had not been 
processed or fully described in the Archives 
Descriptive Database system. 

@ Although the Archives has made good 
progress in developing a modern new storage 
facility for archival records, the storage facili- 
ties and storage methods for the more current 
semi-active records that are destined for the 
Archives pose a risk of deterioration or loss of 
these records. 

We made a number of recommendations for 
improvement and received commitments from the 
Archives that it would take action to address our 
concerns. 


Status of Recommendations 





According to information received from the 
Archives, progress is being made in implementing 
most of our recommendations. Given the magni- 
tude of the actions required, substantially more 
time will be needed for the Archives to fully imple- 
ment or address several of our recommendations, 
such as the recommendations that involve dealing 
with the large quantity of archival electronic rec- 
ords, reducing the growth in the volume of stored 
records, and fully describing archival records in the 
Archives Descriptive Database system to make them 
more accessible to the public. 

The status of the action taken on each of the 
recommendations is described in the following 
sections. 


RECORD-RETENTION SCHEDULES 


Recommendation 1 
To ensure that records created by all program areas of 
ministries and agencies are assessed for their archival 
value and the length of time they should remain in 
storage, the Archives, in collaboration with ministries 
and agencies, should: 
® complete its analysis of each ministry and 
agency, establish a list of all program areas that 
are required to prepare record-retention sched- 
ules, and periodically update that list; and 
® complete its system enhancements so that it can 
ensure that it obtains and authorizes record- 
retention schedules from all those required to 
provide one. 


OVERSEEING OF MINISTRY AND AGENCY 
RECORD-MANAGEMENT PRACTICES 


Recommendation 2 
To ensure that the Archives can fulfill its obligations to 
monitor compliance by ministries and agencies with 
record-management requirements, it should: 
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e@ establish a cost-effective means of periodically 
obtaining the information it needs to monitor 
ministry and agency compliance; and 

@ use this information to identify best practices 
among ministries and agencies and address any 
gaps between the directive, Archives policies 
and guidelines, and actual record-management 
practices. 


Status 
The following covers the status of both Recommen- 
dations 1 and 2. 

The new Archives and Recordkeeping Act that was 
implemented in 2007 strengthened the responsibil- 
ities of both the Archives and Archivist and the min- 
istries and agencies. The Archives informed us that 
since that time, it has been working with ministries 
and designated provincial Crown agencies to mod- 
ernize records management across government. 

A multi-year strategy has been put in place for 
the Archives to complete its high-level analysis of 
each ministry’s and agency’s creation of records 
with archival value and record-retention require- 
ments. The Archives plans to complete all analyzes 
by the end of the 2010/11 fiscal year. It intends to 
use these analyzes in monitoring ministries’ and 
agencies’ compliance with the Act. The Archives 
advised us that as of March 31, 2009, it had com- 
pleted or updated 46% of ministry analyzes and 
48% of agency analyzes. 

In April 2008, the Archives streamlined the 
requirements and procedures for the approval of 
record-retention schedules. The Archives has estab- 
lished a team of six records advisors that is provid- 
ing advice and training to ministries and agencies 
on record-retention schedules and other related 
issues. According to the Archives, as a result of an 
increase in requests from ministries and agencies to 
update and add record-retention schedules, more 
than 2,000 updates were made in the 2008/09 
fiscal year. In addition, more than 700 obsolete or 
superseded record-retention schedules have been 
deleted to date. 


In December 2008, the Archives contacted all 
ministries asking for confirmation on their use of 
updated record-retention schedules for common 
records, such as human resources and finance. Over 
half of ministries had responded to the Archives as 
of June 2009, with responses from all ministries 
expected by September 2009. The Archives plans to 
make a similar request to agencies before the end of 
the 2009/10 fiscal year for completion by Septem- 
ber 2010. The Archives plans to issue compliance 
reports on record-retention schedule coverage for 
individual ministries by December 2009 and for 
agencies by December 2010. Analysis of inactive 
record-retention schedules will also be completed 
for ministries and agencies by these dates. 

The Archives has converted approximately 
14,000 approved record-retention schedules 
(formerly paper-based) to an electronic database. 
On-line access to the database and an information 
resource site for records management is available 
to ministries on the government intranet. We were 
informed that the agencies are to be provided with 
on-line access to the database in the fall of 2009. 

Subject to Management Board of Cabinet 
approval, the Archives was planning to publish a 
government-wide standard for storage of semi- 
active paper records by September 2009. 


ELECTRONIC RECORDS 


Recommendation 3 
In order for the Archives to oversee, manage, and 
archive electronic documents and records created by 
ministries and agencies, the Archives should: 

@ ensure that it has the necessary technical exper- 
tise and capacity to deal with electronic records; 
and 

@ then establish and implement strategic plans 
that would permit the efficient transfer of 
archival electronic documents and records to 
the Archives in accordance with recognized best 
practices for information management. 
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Status 
The Archives continues to rely on the govern- 
ment’s information technology (IT) community 
for preserving and maintaining electronic records. 
According to Archives estimates, by 2013 approxi- 
mately 144 terabytes of ministry and agency 
electronic records will need to be preserved by the 
Archives. To put this amount in perspective, it is the 
equivalent of approximately 3.6 million containers 
of paper-based records—that is, nine times larger 
than the Archives’ current holdings of 0.4 million 
containers. The Archives noted that finding a solu- 
tion to the preservation of electronic records has 
been identified as a priority in Ontario’s five-year 
Corporate I&IT Plan (2008-2013), and it is also rec- 
ognized as a key priority in the Archives’ Strategic 
Plan for 2007-2010. 

In March 2009, a project to establish archival 
processes and procedures for arranging, describing, 
and providing access to electronic archival records 
was completed. These processes and procedures 
were tested using a collection of electronic records 
as a pilot. 

The Archives informed us that it has conducted 
research to examine various national and inter- 
national programs being undertaken to preserve 
electronic records and make them accessible. 
According to the Archives, its research suggests that 
it should base its program on the Open Archival 
Information System model developed by the Inter- 
national Standards Organization (ISO). 

A multi-year project is to start in the 2009/10 
fiscal year to establish a trusted digital repository 
based on the ISO standards. Such a large-scale 
enterprise-wide project will require obtaining fund- 
ing and approval for development of functional 
requirements for the repository, and for analyzing 
the marketplace to identify suppliers of the com- 
ponents needed for the repository. 


GROWTH IN VOLUME OF STORED 
RECORDS 


Recommendation 4 
To fulfill its obligations to manage the growth in the 
volume of stored government records, the Archives 
should, in collaboration with ministries and agencies, 
develop strategies and timetables for reducing the 
growth in and minimizing the volume of records that 
require storage. 

In order to manage the expected significant growth 
in the number of records destined for the Archives and 
to ensure that the Archives can manage its operations 
cost-effectively in future, the Archives should identify 
and accept only those records that clearly have perma- 
nent and historical significance. 

To ensure that records are not being stored longer 
than they need to be, the Archives should determine 
the number and types of records that remain in ISR 
storage facilities past their originally authorized 
retention dates or are unaccounted for and the 
reasons for the delays in their disposition. It should 
use this information to evaluate its policies and pro- 
cedures and those of ISR with the objective of reducing 
any unnecessary or prolonged storage of records and 
delays in transferring archival records. 

In addition, the Archives should investigate min- 
istries and agencies that have not returned records 
that they retrieved from ISR and should implement 
controls that will ensure that unreturned records are 
followed up. 


Status 
Standardized record-retention schedules for 
transitory records (such as records of temporary or 
minor transactions having no ongoing value—for 
example, memos or messages) were issued to min- 
istries and agencies in 2008, making it clear that 
duplicate copies and records of short-term value 
can and should be discarded quickly. The transitory 
record-retention schedules allow paper records to 
be discarded if the same information is maintained 
in electronic form. The Archives informed us that 
following this policy change, it observed a decrease 
in the proportion of revised record-retention 
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schedules submitted for approval that specified 
retention of paper as well as electronic versions of 
information. 

We were advised that the work done so far and 
planned by the Archives to promote modern record- 
Management practices, such as its efforts to ensure 
that ministries’ and agencies’ record-retention 
schedules are up-to-date, is likely to result ina 
modest reduction in both the length of time records 
are stored at ISR and the proportion of records 
transferred to the Archives. The adoption of the 
standardized record-retention schedules was still 
in progress, and given the natural time lags in the 
transfer of records, it may be some time before the 
impact in reducing records will be seen at ISR or 
the Archives. 

The Archives continues to review incoming 
transfers of records that have reached their dispos- 
ition date at ISR and to apply selection strategies 
where possible to reduce the overall quantity of 
records held as archival. We were informed that as 
a result of its review process, the Archives reduced 
by over 30% the volume of records originally sched- 
uled to be transferred to the Archives during 2008. 

In late 2009, the Archives was planning for a 
review of the retention and appraisal policies for 
court records, which account for a large volume of 
the paper records that are destined for the Archives 
but stored in the interim at ISR warehouses. 

The Archives has developed new policies 
and procedures for monitoring, controlling, and 
reducing the volume of records placed on hold by 
ministries and agencies in ISR warehouses past the 
originally established destruction date. A tracking 
database has been implemented to document a 
ministry’s or agency's rationale for the need to store 
the records and for the recommended hold period. 
This approach allows for more timely follow-up 
when the hold period expires. Annually, the new 
process requires a ministry’s program area to pro- 
vide a rationale (for example, ongoing litigation) 
for requests to hold records past their designated 
destruction date, as well as an indication of how 
long they need to be held. The Archives informed 


us that the number of yearly destructions increased 
approximately 40% between 2006 (before the new 
process) and 2008. 

As well, the Archives informed us that it has 
established a process to help ministries and agen- 
cies identify any records that they have retrieved 
from ISR warehouses but not yet returned. Each 
ministry and agency has received a report identify- 
ing all outstanding unreturned records, which will 
assist them in locating and returning the records. 
Subsequently, ISR sends monthly reports identify- 
ing retrieved records that have been outstanding 
more than 90 days. Once a year, ISR flags all 
unreturned records. The Archives noted that most 
ministries are using the reports to better manage 
their ministry’s retrievals process and are providing 
information on files that are not being returned 
because of operational use. The Archives claims 
that there was a significant drop in files that remain 
out—from more than 500,000 before the reporting 
process was initiated to 380,000 as of March 31, 
2009. 


INVENTORY CONTROL FOR THE 
ARCHIVES 


Recommendation 5 
To ensure that proper and effective inventory controls 
are established and maintained for archival records 
and collections and to reduce the risk of loss and theft, 
the Archives should conduct, possibly with the assist- 
ance of ministry internal auditors or other experts, 
a thorough assessment of its inventory and security 
controls and other loss-prevention measures and cor- 
rect any deficiencies identified. 


Status 
We noted that the Archives has made some progress 
in improving its inventory and security controls. 
The Archives informed us that it has completed 
its project to bar-code all of its remaining contain- 
ers of records and enter those records into its 
electronic container-tracking system. More than 
48,000 entries were added to the system during the 
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2008/09 fiscal year in anticipation of the Archives’ 
March 2009 move to a new head-office facility. The 
Archives’ current process is to capture new records 
in the tracking system as they are acquired. 

However, records inside containers (such as 
photographs and documents) are still not cata- 
logued at the item level. As a result, the Archives’ 
inventory systems cannot provide complete details 
of the records inside the containers, including 
whether any of the records are missing. The 
Archives advised us that cataloguing at the item 
level is not feasible given the large volume of docu- 
ments and its limited resources. However, it expects 
that its enhanced security at its new head-office 
public facilities will minimize the risk of theft. 
Furthermore, we were informed that, although the 
Archives began sealing containers during transit 
in 2007, sealing containers when in storage is not 
being considered for the government-operated or 
privately operated storage facilities, because the 
Archives did not feel that this security measure was 
practical. Instead, it planned to continue to rely on 
the security measures already in place at the facili- 
ties. We were advised that the facilities’ security 
measures, such as ensuring that doors are locked, 
are tested during routine inspection of storage 
facilities carried out by Archives staff. 

The Ministry’s internal auditors were engaged to 
help assess the physical security and environmental 
risks, such as fire and flood, of the Archives’ new 
head office and to help develop policy and proced- 
ures to support secure access, and the Archives 
informed us that it has acted on recommendations 
made by the internal auditors. We were advised 
that only retrieval staff now have access to vaults 
containing certain valuable archival holdings. How- 
ever, we noted that the Ministry’s internal auditors 
were not asked to help assess security and inventory 
controls and risks for records and archival holdings 
at ISR and Archives storage warehouses. 

A province-wide inventory and secure mounting 
of artworks in the government of Ontario art collec- 
tion were in progress at the time of our follow-up. 
The Archives put a procedure in place in March 


» 


2008 to follow up on artworks that are reported as 
unaccounted for or stolen. We were advised that 
since the procedure was put in place, there have 
been no instances of unaccounted-for or stolen art. 


ACCESS TO ARCHIVAL COLLECTIONS 


Recommendation 6 
In order to improve access to archival collections, the 
Archives should identify records that have not been 
listed or described fully in its Archives Descriptive 
Database system and should establish a plan and 
timetable for their inclusion. 


Status 

We were advised that a project to review the rec- 
ords not listed or fully described in the Archives’ 
on-line catalogue, called the Archives Descriptive 
Database, was under way at the time of our follow- 
up. As part of the Collections Management Backlog 
Reduction Strategy that the Archives established in 
the 2008/09 fiscal year, the Archives has developed 
selection criteria for prioritizing backlog projects to 
be added to the database. This multi-year strategy 
identified that approximately 60% of non-govern- 
ment collections and 33% of government records 
required some degree of processing to be complete. 

The Archives informed us that it no longer adds 
to the backlog. All newly acquired collections, 
whether private records or government records, 
are to be fully described in the Archives Descriptive 
Database system within a year of receipt in order 
to ensure that the backlog is capped at its current 
level. We were advised that, as of the end of March 
2009, about 88,000 updates had been entered in 
the database and made publicly accessible. This 
reflected the processing of 57 recent acquisitions, 
the archival records transferred from ISR ware- 
houses during 2008, and 15 high-priority backlog 
collections. 

Work was to continue on the backlog but we 
were advised by the Archives that it could take 
10 years to complete, depending on the Archives’ 
available resources and competing operational 
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priorities. In the interim period, backlog collections 
will be made physically available to the public and 
are listed in the container-tracking system. 


STORAGE FACILITIES 


Recommendation 7 
In order to better protect and preserve records that 
are destined for the Archives and that are normally 
not stored in facilities with suitable environment and 
building conditions, the Archives should: 

@ conduct a cost-benefit and feasibility analysis to 
determine if it should make greater and earlier 
use of its environmentally sound storage facility; 

@ examine ISR’s storage policies, procedures, and 
facilities to identify changes that would improve 
the environment and building conditions for 
records in their facilities that are scheduled to be 
transferred to the Archives; and 

e evaluate and make improvements to its inspec- 
tion program and reporting requirements for 
ISR’s storage facilities and establish policies and 
procedures for requiring corrective action when 
inspection results are unsatisfactory. 

In addition, the Archives should ensure that 
government of Ontario artworks are protected and 
preserved by storing them in environmentally sound 
and secure facilities. 


Status 
The Archives informed us that it was in the process 
of considering options for its long-term storage 
facilities requirements for semi-active records, 
which are currently met by ISR’s two government- 
owned-and-operated warehouses (which hold 
about 400,000 containers) and by a private-sector 
company’s warehouse facility (which holds about 
600,000 containers). A business case and cabinet 
submission for approval were planned for during 
the 2009/10 fiscal year. In the interim, as a result 
of a tender, a new three-year agreement with its 
private storage-facility provider was established 
in January 2009, with options by the Archives for 
three one-year extensions to the term. 


The new agreement with its private storage- 
facility provider includes stricter and clearer 
requirements for the storage environment. Informa- 
tion on temperature and humidity is being cap- 
tured and monitored at all warehouses. We were 
informed that quarterly environmental inspections 
of the private storage facility are being conducted. 
Also, a contracted company was hired to provide 
the Archives with monthly reports that outline the 
results of environmental conditions at each ware- 
house facility, whether government-operated or 
privately run. 

We were advised that water-detection sensors 
have been installed in government-owned 
warehouses, and precautions have been taken to 
eliminate the risk of flood damage to containers 
stored at these warehouses. Formal inspections 
have increased to quarterly for government-owned 
facilities and monthly for the private company’s 
facilities. We were advised that issues identified at 
inspections are brought to management’s attention 
for resolution. 

Artworks are no longer permanently stored in 
a basement storage room of a Queen’s Park build- 
ing. Most artworks not on display were moved to a 
secured environmentally controlled storage vault 
at the Archives’ new head office in May 2009, anda 
few remaining artworks were planned to be trans- 
ferred in July 2009. A new Art Collection Tracking 
System was also established, and artworks are to 
be bar-coded and scanned into the new system for 
inventory tracking. 


PRIVACY CONTROLS OVER RECORDS 


Recommendation 8 
In order to ensure that the confidentiality of records 
in storage is protected and that service providers are 
in compliance with the security and confidentiality 
requirements of their contracts, the Archives should: 
e conduct a thorough privacy risk assessment of 
its storage facilities operated by private-sector 
contractors; 
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@ restrict activities and impose security controls at 
storage facilities that will minimize the exposure 
of confidential records; 

@ revise its inspection program of storage facilities 
to include formal assessments of its security and 
privacy controls; and 

e develop classification criteria for confidentiality 
and security levels and establish special storage 
arrangements for the most sensitive records. 


Status 
Guidelines issued during the 2006/07 fiscal year 
by the Information and Privacy Commissioner of 
Ontario and the Chief Information and Privacy Offi- 
cer recommend that when ministries contract with 
private firms, a thorough risk assessment should 
be conducted. The Archives informed us that, 
in response to our recommendation in the 2007 
Annual Report, a privacy risk assessment was under 
way that would examine both the government-run 
and private-sector storage facilities. Although the 
Archives did complete a privacy risk assessment for 
its government-run warehouses in September 2008, 
no privacy assessment had yet been performed 
at the private storage facilities at the time of our 
follow-up. 

The Archives informed us that it has nonethe- 
less completed a project to eliminate personal 
information from the exterior of boxes at both its 
private- and public-sector facilities. Going forward, 
staff at these facilities are required to ensure that 
no personal information is noted on the exterior of 
storage containers. In addition, the Archives’ recent 
contracts for storage and transportation services 
include provisions regarding confidentiality and 
the Freedom of Information and Protection of Privacy 
Act, and monthly inspections of the private storage 
facility include testing security. 

However, we noted that certain control weak- 
nesses we identified in our 2007 Annual Report 
were not addressed in recent contracts awarded 
to the Archives’ private-sector storage contractor. 
First, the Archives advised that, because this is an 
industry-standard practice, it continues to allow its 


private-sector storage contractor’s staff to retrieve 
individual files from storage containers, a practice 
that risks exposing potentially sensitive informa- 
tion. Second, no changes were made to require that 
ISR’s approximately 600,000 containers be separ- 
ated from containers belonging to the contractor’s 
other clients. ISR containers represent about 30% 
of the total holdings at the contractor’s facilities, 
and separating these containers could help to 
reduce the risk of loss or security breaches. 

At the time of our 2007 audit, a government- 
wide file-classification plan was established to 
promote consistency across the government in the 
identification and organization of records and to 
classify information according to its sensitivity. 
However, at the time of our follow-up, the Archives 
had not changed its systems and procedures so that 
it could use this information to designate records 
that should remain in facilities with enhanced 
security arrangements. 


CONTRACT MANAGEMENT 


Recommendation 9 
In order to ensure that contracts with service provid- 
ers are managed properly and that procurement pro- 
cesses are documented properly, ISR and the Archives 
should evaluate the way in which they manage 
procurement documents, and ISR should ensure that 
payments are made only for amounts and services 
that are in accordance with contractual requirements. 


Status 
The Archives informed us that it has implemented 
a document-management system to track procure- 
ment and contract management documentation. 
A Contract Tracking Report is also reviewed and 
updated on a monthly basis. 

Documentation was provided to us for several 
large contracts recently awarded by the Archives 
for transportation services and private storage 
facilities. Procurement decisions in each of these 
cases were well documented and in accordance with 
established policies and procedures. However, the 
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Archivist of Ontario recently asked the Ministry’s 
internal auditors to review a contract for file-storage 
shelving and related services, valued at $775,000. In 
January 2009, the internal auditors concluded that 
this contract, already spent, had been inappropri- 
ately awarded because it had not been competitively 
tendered. The Archives responded to the internal 
auditors that it would comply with the procurement 
policies and procedures in the future. 

Controls for ensuring that the Archives’ private 
storage-facility provider meets contractual require- 
ments for maintaining suitable environmental 
and security controls were also enhanced through 
improvements to inspection procedures and fre- 
quency and to reporting requirements. 


The Archives informed us that, due to weak- 
nesses in its ISR container-tracking system and 
backlogs in data entry regarding storage and con- 
tainer movement activities, it is unable to establish 
procedures for reconciling differences between the 
volumes reported by its container-tracking system 
with its contractor’s monthly billings for storage 
services. We noted that discrepancies continue to 
exist between the information in the ISR’s con- 
tainer-tracking system and its contractor’s invoices, 
which the Archives continues to pay without verifi- 
cation procedures. The Archives was planning for a 
new container-tracking system, which will permit it 
to verify contractor invoices. 
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Section 


Background 


The Centre of Forensic Sciences (Centre) provides 
independent forensic-science laboratory services to 
law-enforcement officers and other justice-sector 
clients. Police investigators and Crown prosecutors 
rely on forensic science to identify or eliminate sus- 
pects and to provide evidence that can withstand 
scrutiny in court. Delays or errors in forensic analy- 
ses can prolong police investigations, increase their 
costs, and affect public safety by allowing criminals 
to remain free to reoffend. 

During the 2008/09 fiscal year, the Centre 
received more than 11,600 requests (10,400 in 
2006/07) from its justice-sector clients for scientific 
analyses of evidence. These requests resulted in the 
issuing of almost 15,100 analytical reports (12,700 
in 2006/07). The types of services (with approxi- 
mate percentage of staff working in each area in 
parenthesis) were as follows: 

e biology (32%); 
toxicology (20%); 
chemistry (14%); 
firearms and toolmarks (8%); 
documents and photoanalysis (4%); and 


electronics (3%). 
The Centre’s head office and central laboratory 
are located in Toronto, and a northern regional 
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laboratory is located in Sault Ste. Marie. During 
the 2008/09 fiscal year, the Centre had operat- 
ing expenses of approximately $26.4 million 
($25.5 million in 2006/07). 

In our 2007 Annual Report, we concluded that 
the Centre had established reasonable processes 
for ensuring the quality of its services, and noted 
that it was pursuing international accreditation in 
this regard for 2008, which it has since received. 
As well, its clients were generally satisfied with the 
calibre of its work. While the timeliness of its ser- 
vices was an issue in the past, over the last several 
years it had improved in this area—its DNA analysis 
in particular—despite an increase of more than 
70% in the demand for forensic services. 

However, improvements in systems and pro- 
cedures were required for the Centre’s turnaround 
times to be comparable to those of leading inter- 
national forensic laboratories. Some of our more 
significant observations were as follows: 

e@ We compared the Centre to two global lead- 
ing forensic-science laboratories—one in the 
United Kingdom and the other in Sweden— 
and found that the two completed their case 
reports in about half the Centre’s average 
turnaround time of 64 days. 

e@ The Centre used only one turnaround-time 
target to monitor the performance of its dif- 
ferent investigative sections. However, since 


the kinds of cases each section works on are 
completely different, each different section 
should have its own turnaround-time target. 

e Having standards and case-completion bench- 
marks by section would facilitate managerial 
oversight and identification of bottlenecks 
requiring corrective action. The Centre’s 
90-day target for completing 80% of its cases 
was much longer than targets set by forensic- 
science laboratories in other jurisdictions, 
which often set targets of 30 days or less. 

e The Centre had established no documented 
systems or procedures for monitoring the 
number of urgent cases processed by each sec- 
tion or their turnaround times. 

e The Centre’s information systems did not help 
management determine why certain case 
reports had been delayed. 

We made a number of recommendations for 
improvements, and received commitments from 
the Centre that it would take action to address our 
concerns. 
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The Centre has made a substantial effort to 

update its systems and procedures to address our 
recommendations and increase its efficiency. Of 
particular note is the Centre’s participation in a 
multi-jurisdictional performance-benchmark pro- 
ject. However, it has so far been unable to improve 
overall turnaround times, given the increased 
demand for its services, and its available resources. 
The status of action taken on each of the recom- 
mendations is described in the following sections. 


MONITORING REPORT TURNAROUND 
TIMES 


Recommendation 1 
In order to ensure that it better meets the needs of its 
clients for investigating and prosecuting crime, the 


Centre of Forensic Sciences 


Centre of Forensic Sciences should conduct a review of 
its practices and resources on an area-by-area basis, 
with a focus on achieving improvements in its turn- 
around times for completing case analyses, especially 
for the more urgent cases. 


Status 
The Centre informed us that it had completed an 
area-by-area review of its practices and resources 
in May 2008, with the objective of improving 
turnaround times. It identified ways it could be 
more efficient through enhancing processes and 
eliminating bottlenecks. These changes were either 
implemented or were under consideration for 
future implementation. 

We were advised that turnaround-time improve- 
ments had been achieved in the biology, chemistry, 
and firearms investigative sections. In the areas 
where turnaround times had not improved, such 
as toxicology, initiatives were underway that were 
expected to improve things over time. In addition, 
the Centre improved its central receiving area’s 
procedures and this section subsequently saw an 
increase in its efficiency. 

However, the Centre was not able to improve 
its overall turnaround times, which have increased 
marginally. The Centre noted, and as Figure 1 
shows, since our audit of two years ago, the Centre 
has had to deal with an almost 11% increase in the 
cases it receives. During this period, it received an 
operating budget increase of 3.5%, yet increased its 
output of completed reports by about 19%. 


Figure 1: Centre of Forensic Sciences Workload and 
Turnaround Times, 2006/07-2008/09 


Source of data: Centre of Forensic Sciences 


% of 

Avg. # Reports 

of Days Completed 

Cases Completed to Issue within 

Fiscal Year Received Reports Reports 90 Days 
2006/07 10,454 12,693 64 79 
2007/08 11,393 14,016 67 73 
2008/09 11,573 15,146 66 79 





347 


Chapter 4 « Follow-up Section 4.02 








348 2009 Annual Report of the Office of the Auditor General of Ontario 





N 
S 
+ 
(he 
Ce 
i 
o | 
® | 
wn} 
a 
ae 
= | 
S 
re 
Le | 
eit 
b= 
he 
® 
oe 
a 
cS 
<= 
© | 


. 


SETTING TARGETS FOR REPORT 
TURNAROUND TIMES 


Recommendation 2 
To ensure that the Centre of Forensic Sciences’ target 
turnaround times for completing case analyses are 
meeting the needs of its clients and the administra- 
tion of justice, the Centre should establish processes, 
involving its clients, to: 
® set turnaround-time targets for the various types 
of investigative services its provides, and segre- 
gate these between urgent and non-urgent cases; 
@ assess actual performance against targets; and 
® compare its turnaround times and methods of 
achieving them with those of other jurisdictions. 


Status 
The Centre informed us that it had conducted tar- 
get surveys on turnaround times in June 2008 and 
held focus groups from June to August of the same 
year with special investigation units, police, and 
the Crown to assess client needs for timely delivery 
of analytical reports for both non-urgent (routine) 
and urgent cases. As a result, the Centre decided to 
bring its method for setting turnaround-time tar- 
gets more into line with the methods used in other 
jurisdictions, so that it would better meet its client 
needs. 

New turnaround-time targets for both routine 
and urgent requests were put into place as of Janu- 
ary 2009. For a routine case, the turnaround time 
target became 30, 60, or 90 days, depending on 
the seriousness of the offence, the complexity of 
the analysis, and the capacity within a section. For 
urgent cases, the Centre began to directly consult 
its client to determine the turnaround time needed 
according to the circumstances and the criteria 
established by the Centre, which has resulted in 
“client-driven” turnaround-time targets being set 
for each case. 

The Centre has also been preparing quarterly 
reports for each investigative section on the per- 
formance achieved against the targets set for both 
routine and urgent cases. 


TRACKING CASES BY PRIORITY 


Recommendation 3 
The Centre of Forensic Sciences should ensure that its 
information systems capture information on urgent 
cases that allows the monitoring and assessment of: 
@ each investigative section’s success in responding 
to urgent cases; 
e the impact of urgent cases on each investigative 
section’s workload; and 
e the turnaround times achieved. 


Status 
The Centre informed us that it had modified its 
information system in November 2008 so that it 
would be able to capture and track performance on 
urgent cases. The system can now record the turn- 
around time committed to the client and the actual 
completion date, and can capture the reasons for 
urgency. 

The impact of urgent cases on each section 
workload is to be assessed annually as part of the 
Centre’s operational and performance-planning 
activities. 

As previously noted, among the Centre’s quar- 
terly reports are those that assess performance for 
urgent cases against the turnaround times to which 
the Centre committed. 

The Centre told us that the data it had collected 
in the first five months since modifying its informa- 
tion system indicated that 89% of all urgent cases 
had been completed by the due date and within an 
average turnaround time of 10 days. 


MONITORING CAUSES OF DELAYS 


Recommendation 4 
To ensure that the causes of delays in processing cases 
are monitored and assessed so that any systemic 
issues can be addressed, the Centre of Forensic Sci- 
ences should: 
@ ensure that its information systems record the 
reasons for any significant delays in each case it 
investigates; 


@ set standards for the processes used by each 
investigative section and monitor variances 
between expected and actual times; 

@ conduct regular evaluations where delays in 
completing cases appear high to identify the 
reasons and determine what steps can be taken 
to mitigate the likelihood of the same delays 
arising in the future. 


Status 
The Centre informed us that it had modified its 
information systems in November 2008 to allow 
section managers to monitor any delays in complet- 
ing cases and record the reasons for delays. 

The system provides reports to managers on 
those cases that are approaching their targeted 
turnaround times. Managers have been reviewing 
these reports and, when necessary, expediting the 
cases to prevent delays. 

The Centre informed us that it had started 
requiring managers to input an explanation into the 
information system for any cases that exceed their 
targeted turnaround time by 50% or more. The cat- 
egories used to describe reasons for delay include 
equipment problems, staffing issues or absences, 
quality incidents, competing workloads, adminis- 
trative bottlenecks, subcontractor issues, supply 
chain problems, insufficient case information, 
inability to contact client, and delays in other sec- 
tions. Analysis of the data collected since November 
2008 has shown the most common cause for delay 
was competing workloads due to large volumes of 
work within a section. 

The Centre informed us that it needed to con- 
tinue collecting data for at least a year to properly 
identify trends in delays. It expected that managers 
would use this information for their next annual 
operational and performance-planning activities to 
identify what caused delays and the necessary cor- 
rective action. 


Centre of Forensic Sciences —En 


MEASURING PERFORMANCE 


Recommendation 5 
In order to better monitor and report on its financial 
and operational performance, the Centre of Forensic 
Sciences should: 

@ establish measures to monitor the cost effective- 
ness of its operations; 

@ benchmark its performance against that of 
other forensic laboratories; 

@ investigate whether its quarterly reports on 
average turnaround times are reaching those 
clients who would best benefit from them and 
consider distributing these reports directly to 
them. 


Status 
The Centre informed us that it had conducted a 
cost-measuring and comparison exercise using 
2007/08 data in February 2009. It has been meas- 
uring cost-effectiveness by calculating each report’s 
average cost, using staffing and other costs in each 
section along with a prorated portion of all other 
support costs. This measurement is to continue 
each year as part of the Centre’s annual operational 
and performance-planning activities. Measures for 
additional activities are to be created by the end 
of 2009/10, after the required changes have been 
made to the Centre’s information systems. 

The Centre also made changes to bring its 
activities more into line with those of other jurisdic- 
tions. It started classifying its cases using industry- 
standard crime-report violation codes and has been 
using internationally recognized definitions to 
record specific examination activities for each case. 

The Centre informed us that, since 2007, 
it has been participating in an ongoing multi- 
jurisdictional benchmarking project. The project is 
led by West Virginia University, sponsored by the 
U.S. National Institute of Justice, and 14 forensic 
labs from across North America are participants, 
including the RCMP. Assessment strategies are to be 
developed by the end of 2009, once enough data for 
comparison and benchmarking have been collected 
from various jurisdictions. 
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According to the Centre, the benchmarking 
exercise is aimed at defining and standardizing 
performance measures that can be used to com- 
pare performance across jurisdictions and assess 
resource allocation, efficiencies, and value for 
services. Some of the key reporting measures to be 
benchmarked for comparison across jurisdictions 
are: 

e breadth of analysis, such as the average num- 

ber of tests conducted per case; 

@ accuracy and quality control, such as the aver- 
age number of tests per item within a case; 

e efficiency and productivity, such as the aver- 
age number of items per full-time equivalent 
staff; 

® market conditions, such as the average salary; 

e labour productivity, such as the average num- 
ber of tests per full-time equivalent staff; 


e economy of operation, such as the average 
cost per case; and 
e other measures, such as backlog of cases older 
than 30 days. 
The Centre informed us that it has continued 
its practice of providing information sheets to its 
clients upon receipt of each case. The client infor- 
mation sheets include information on targeted and 
actual turnaround times, which are updated ona 
quarterly basis, and are intended to give the Cen- 
tre’s clients a reasonable expectation of when they 
will receive the results for their cases. 
As part of its annual client satisfaction survey, to 
be conducted in 2009, the Centre plans to include a 
question seeking clients’ feedback on the usefulness 
of the performance reports, such as client informa- 
tion sheets on turnaround-time statistics, that they 
received. 







Section 


Background 


The Developmental Services Program of the Min- 
istry of Community and Social Services (Ministry) 
funds community-based agencies that provide a 
broad range of services and support for both adults 
and children with a developmental disability. In the 
2008/09 fiscal year, expenditures on this program 
were approximately $1.5 billion ($1.2 billion for the 
2006/07 fiscal year), almost two-thirds of which 
went to the Community Accommodation Program 
to provide residential accommodation and sup- 
port services for both children and adults with a 
developmental disability. 

The Ministry’s expenditures on the Community 
Accommodation Program at the time of our audit 
in 2007 were $767 million, more than double 
what they were at the time of our previous audit 
of the program in 1999, and grew to approxi- 
mately $976 million for the 2008/09 fiscal year. 
The largest portion of these expenditures was for 
adult-group-home accommodation. Residential 
placements are based on the assessed needs of the 
individual and range from relatively independent 
living arrangements in apartment-like settings with 
regular agency support to intensive 24-hour-a-day, 
seven-day-a-week care in group homes typically 
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housing three to six individuals. However, access to 
residential services is limited by the availability of 
existing spaces, which are primarily dependent on 
ministry funding. 

In our 2007 Annual Report, we made a number 
of observations that were similar to those we had 
made in our 1999 audit. The most significant of 
these observations were as follows: 

e For many years, agency funding has primarily 
been historically based rather than needs- 
based, which exacerbates funding inequities. 
Agency budget submissions lack the suffi- 
ciently detailed information required to make 
informed funding decisions, and there is little 
evidence that budget submissions have been 
reviewed and assessed for reasonableness. 
Many agencies did not receive their final 
approved budget until long after the fiscal 
year had ended. 

e The annual budgeting process left the 
Ministry without the ability to monitor or 
compare information such as the average cost 
of spaces and services within a home. Costs at 
the agencies we visited ranged from $30,000 
to more than $200,000 per person per year, 
according to our calculations. The Ministry 
was unaware of these cost differences and was 
unable to demonstrate that they were reason- 
able or justified. 
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@ The Ministry lacked the necessary procedures 
and expertise to ensure that it is receiving 
value for money for the capital projects it 
funds, and some of the costs incurred seemed 
excessive. In one instance, it spent $380,000 
to renovate a bungalow that had been pur- 
chased for $390,000—without assessing the 
need for and reasonableness of the renovation 
or receiving a proper accounting of the costs. 

e There was often little documentation to sup- 
port an individual’s developmental disability 
determination or to demonstrate that the 
placement of an individual was appropriate 
and cost-effective. 

e We noted a number of instances where beds 
remained vacant for six to 12 months. The 
Ministry’s funding mechanism pays equally 
for vacant and occupied beds, leaving the 
agencies with little incentive to fill vacant 
beds. At the same time that beds in some 
agencies remained vacant for extended 
periods, those agencies’ access centres had 
lengthy waiting lists for accommodations. 

We made a number of recommendations for 
improvement and received commitments from the 
Ministry that it would take action to address our 
concerns. 


Status of Recommendations 





According to information obtained from the 
Ministry of Community and Social Services and a 
review of work undertaken by the Ministry’s inter- 
nal audit division, some progress has been made 

in addressing most of our 2007 recommendations, 
but more needs to be done to address all areas satis- 
factorily. The status of action taken on each of our 
recommendations was as follows. 


AGENCY GOVERNANCE AND 
ACCOUNTABILITY 


Recommendation 1 
To help ensure that transfer payments to agencies 
represent value for money spent and that services pro- 
vided are effective and in accordance with program 
requirements, the Ministry of Community and Social 
Services and its transfer-payment agencies should 
adhere to the mandatory governance and account- 
ability frameworks. 


Status 
The Ministry informed us that it was implementing 
an agency governance strategy that included a four- 
point plan as follows: 

e clarify ministry expectations for good agency 

governance; 

e include the clarified governance expectations 

in the service contract; 

® monitor and report on how agencies meet 

expectations on governance; and 

e develop progressive corrective action for 

medium- and high-risk agencies. 

The Ministry also informed us that it was draft- 
ing a guide on basic requirements for transfer 
payment accountability, which was to be sent to 
agencies once it was finalized. It has provided train- 
ing to ministry staff and developed a learning guide 
that includes practical guidance on accountability. 
The Ministry also made a number of amendments 
to improve the clarity of its standard service 
contract. These amendments included a note 
on the Ministry’s expectations regarding agency 
governance. 

The most significant improvement in this area 
since our 2007 Annual Report was the development 
of an agency risk assessment process that looks at 
eight different areas, including agency governance. 
The Ministry has already identified medium- and 
high-risk agencies using the process and will 
require these agencies to submit action plans 
outlining how risk will be mitigated and corrective 
action taken. At the time of our follow-up, however, 
the Ministry informed us that there had not been 


sufficient time to review agency progress to address 
the issues that had been identified. 


TRANSFER PAYMENT CONTROLS 


Budget Submissions and Annual Service 
Contracts, Quarterly Reports, Annual 
Program Expenditure Reconciliation, Staff 
Qualification and Training. 


Recommendation 2 
To ensure that funding provided to service-delivery 
agencies is based on assessed needs and is commensur- 
ate with the value of the services provided, and to 
implement more effective financial accountability in 
transfer payments to agencies, the Ministry of Com- 
munity and Social Services should: 

e reassess the objectives of its annual agency 
budget submission, review, and approval pro- 
cess, and design a meaningful process that it can 
adhere to; 

e either implement its current quarterly reporting 
process effectively or design and implement 
a revised process that it can adhere to and 
that will enable regional staff to monitor in- 
year agency expenditures and service levels 
effectively, possibly screening agencies on a few 
critical indicators; 

@ assess whether its current APER process as 
implemented meets its objectives and, if it does 
not, design a more practical means of overseeing 
agency expenditures; and 

e assess the level of financial expertise required in 
regional offices, and determine the number of 
staff with this expertise that it requires and the 
best way of acquiring this expertise. 


Status 
The Ministry has made some improvements to 
the annual agency budgeting process, quarterly 
reporting, and annual financial reporting processes. 
These changes were being piloted at the time of 
this follow-up, with full implementation to occur in 
the 2010/11 fiscal year. The Ministry approved an 
information strategy that includes improvements as 
follows: 
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e Standardized categories of financial informa- 
tion on revenues and expenditures, as well as 
for staffing, are now required for budgeting 
and quarterly reporting purposes, as opposed 
to the variety of information that was previ- 
ously received. The goal of these new require- 
ments is to ensure better information for 
cost—value analysis, to compare performance 
metrics across agencies, to be able to perform 
regional variation analysis, and to simplify 
reporting. 

e@ It streamlined the number of expenditure 
categories used for financial reporting by half 
to provide more accurate and meaningful 
information about the services provided. 

@ It developed business requirements for the 
automation of the transfer payment budget 
package. Pending development of this auto- 
mated solution, the Ministry developed a 
modified budget package and a central reposi- 
tory that stores all budget and quarterly data, 
financial and data standards, and validation 
rules to improve the accuracy of data-entry 
done by agencies. 

The Ministry informed us that it had also imple- 
mented a new annual reconciliation process called 
Transfer Payment Annual Reconciliation (TPAR), 
which replaced the Ministry’s Annual Program 
Expenditure Reconciliation (APER) during the 
2007/08 fiscal year. The goal of the new process is 
to simplify the reconciliation process and provide 
the Ministry with better information on financial 
performance. It includes new reconciliation and 
verification requirements, the submission of seg- 
mented information by specific program, such as 
residential or individualized living costs, along with 
audited financial statements and the requirement 
to submit a post-audit management letter. Training 
on TPAR was provided to both ministry and agency 
staff in 2008. 

With regard to staff qualifications, the Ministry 
has not yet reviewed and assessed the need for 
financial expertise in its regional offices, but plans 
to do so in the near future. However, the Ministry 
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has implemented a learning and development 
curriculum to enhance the knowledge and skills 
of existing staff. At the time of this follow-up, 19 
training sessions had taken place. In addition, the 
Ministry set up on-line resources, including self- 
directed learning, to assist staff. 

Although we have noted these improvements, 
the Ministry’s internal audit testing identified lack 
of timely budget approvals and reporting as an 
outstanding issue. Budget packages continued to be 
sent out late to agencies, and agency submissions 
of critical financial information were still not being 
received or approved by the Ministry on a timely 
basis. 


Facilities Initiative 


Recommendation 3 
In order to ensure that funding given to agencies for 
relocating persons from provincially operated facili- 
ties into community placements is reasonable and 
appropriate, the Ministry of Community and Social 
Services should: 
@ assess the merits of instituting a more com- 

" petitive process instead of having community 
planning tables (committees consisting of rep- 
resentatives of local service agencies) nominate 
only one agency to submit a proposal for placing 
an individual in the community; and 

e obtain sufficiently detailed budgetary or other 
information for assessing and documenting 
the reasonableness of the amount of funding 
requested where that amount differs signifi- 
cantly from the expected cost amount. 


Status 
The movement of individuals from provincially 
operated facilities into community placements was 
completed by March 2009. Although the Ministry 
did not implement more competitive processes for 
determining placement of individuals into the com- 
munity as was recommended, it informed us that 
the regional offices had implemented strategies 
to ensure the reasonableness of budgets for place- 
ments. In particular, regional offices were providing 


only first-year funding to agencies whose place- 
ment costs exceeded the established benchmarks 
and were not committing to ongoing future years’ 
annualized funding until a detailed review of costs 
had been completed. Third-party reviews of costs 
were also conducted in some regions to determine 
the reasonableness of any placement costs that 
exceeded the established benchmark. 

These strategies notwithstanding, the Ministry 
has not demonstrated that service and funding 
decisions regarding these placements were reason- 
able and appropriate, and funding has continued 
to be approved without sufficiently detailed 
information to support the cost of the placements 
submitted. 


Oversight of Capital Projects 


Recommendation 4 
In order to be able to demonstrate that capital funding 
provided to agencies is both necessary and reasonable 
and creates as many spaces as possible, the Ministry 
of Community and Social Services should: 

e ensure that all existing requirements in its 
Guidelines for Developing Infrastructure are 
complied with, including the requirement to 
prepare a business case that demonstrates that 
alternatives were considered and that the most 
cost-effective alternative was selected; 

@ obtain the necessary expertise (by engaging an 
external expert if necessary) and sufficiently 
detailed information for all proposed projects 
to be able to assess the need for, and reasonable- 
ness of, the costs to be incurred; and 

e obtain a final accounting of the costs incurred. 


Status 


The Ministry hired four capital analysts to assist the 


regions in ensuring that critical documentation was 
on file for all capital projects completed at the time 
of our 2007 audit and to provide ongoing support to 
regions for capital projects relating to the facilities 
initiative. One capital analyst was to be retained 
after the completion of the facilities initiative. 


The Ministry updated its guidelines in 2007 
following the completion of our audit work and 
informed us that at the time of our follow-up a fur- 
ther update was in progress. It had also developed 
new tools to support the regions and transfer pay- 
ment agencies. The Ministry has been providing 
training on capital processes and procedures to the 
regions. 

The Ministry’s internal audit division reviewed 
capital expenditure files at the regions and noted 
that files were generally well organized and sup- 
ported the need for the capital project. However, 
more emphasis was needed on documenting the 
alternatives considered and competitive bids 
received to demonstrate that the most cost-effective 
option was selected. The internal audit division also 
noted that in some cases supporting invoices and 
other necessary information to account for the final 
costs incurred for capital projects were missing and 
that when capital agreements were amended, the 
reason for the changes was not explained. 


Agency Purchasing Policies and 
Procedures 


Recommendation 5 
To help ensure that agency expenditures are reason- 
able and represent value for money spent, the Ministry 
of Community and Social Services should confirm 
that agency boards of directors ensure adherence to 
good business practices, including written policies and 
procedures for such things as purchasing goods and 
services and processing invoices for payment. 


Status 
The Ministry informed us that it had issued “best 
practices” to its transfer payment agencies in Nov- 
ember 2008 in the following areas: 

e travel, meal, and hospitality expense 
reimbursement; 

e the procurement of goods, services, and con- 
sulting services, and the use of agency credit 
and purchasing cards; and 

@ fleet management and the use of other road 
transportation. 
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The Ministry has indicated to its agencies that 
it expects them to incorporate these best practices 
into their policies and procedures, and requires 
them to provide written confirmation of their 
compliance with them. The Ministry has also 
advised the agencies that they will be expected to 
demonstrate their compliance with the best practi- 
ces when they are next audited by the Office of the 
Auditor General of Ontario. 


OVERSIGHT OF PROGRAM SERVICE 
DELIVERY 


Access to Services 


Recommendation 6 
To help ensure that all individuals with a develop- 
mental disability are treated consistently across the 
province and that program placements are appropri- 
ate and economical, the Ministry should: 

@ consider providing access centres with guidelines 
to encourage consistent placement decisions 
across the province; 

@ ensure that access centres maintain the neces- 
sary documentation to demonstrate that 
developmental disability determinations are 
made consistently and that residential place- 
ments are appropriate and economical; 

e ensure that all vacancies are filled as quickly as 
possible; and 

e obtain information about waiting lists and 
vacant beds for use in its service planning pro- 
cess and take this information into considera- 
tion when making annual funding decisions. 


Status 
The Ministry informed us that it had piloted a new 
application form and common assessment tool, 
which had yet to be implemented across the prov- 
ince. The new application form incorporates a set of 
risk factors that is to be part of the new prioritiza- 
tion process. At the time of this follow-up, the risk 
factors still needed to be validated to provide a con- 
sistent approach to managing the various waiting 
lists for developmental services across the province. 
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The Ministry also informed us that, as a result 
of new requirements for service contracts, agencies 
are required to report vacancies to local access cen- 
tres within 10 days of the vacancy occurring. Access 
centres in turn are required to maintain data on the 
number of vacancies by agency and program type, 
and to provide an explanation to the Ministry for 
any vacancy not filled within the quarter. 

Proposed legislation outlining a revised pro- 
cess for access to services has been introduced 
since our 2007 audit. The legislation designates 
placement and funding entities that will have the 
authority to administer the application process. 
Once designated, these entities will be responsible 
for prioritizing access to services, establishing 
and maintaining waiting lists, and issuing annual 
reports to the Ministry. The Ministry will be 
required to publish these reports within 60 days of 
receipt. It is expected that the proposed legislation 
will be proclaimed in the 2010/11 fiscal year and 
the entities are to assume their new responsibilities 
in the 2010/11 and 2011/12 fiscal years. 


Program Compliance Reviews and 
Licensing Inspections 


Recommendation 7 
To help ensure that the Ministry’s compliance review 
process meets its objective of protecting vulnerable 
people in care, the Ministry of Community and Social 
Services should: 

® maintain an accurate and up-to-date listing 
of all adult group homes and ensure that the 
requirement to review 20% of them annually 
is met, and that higher-risk group homes are 
reviewed with reasonable frequency; 

@ reassess the advisability of having agencies select 
the homes to be reviewed and of giving signifi- 
cant advance notice of reviews; 

@ consider developing a comprehensive checklist 
that would help ensure that all the required ele- 
ments of the compliance review are undertaken 
and adequately documented; and 


@ ensure that where deficiencies are identified, 
they are followed up on to confirm that the 
necessary corrective action is taken in a timely 


manner. 


Status 
The Ministry informed us that it had made improve- 
ments in the area of compliance reviews since 
the time of our 2007 audit. These improvements 
included the development of a standardized, 
automated checklist to be used for compliance 
reviews and the creation of an up-to-date and 
accurate list of group homes funded under the com- 
munity accommodation program, which is to be 
maintained by the Ministry’s Service Management 
Information System and its Automated Licensing 
System. 

The Ministry also established a combined cor- 
porate/regional review model in 2008 to conduct 
compliance reviews of adult group homes. This 
two-year project is to be completed in 2010. Its 
goal is to bring compliance reviews up to date and 
establish processes to maintain the currency of 
compliance inspections and the list of eligible group 
homes. At the time of this follow-up, approximately 
24% of all eligible adult group homes had under- 
gone compliance inspections. The Ministry has indi- 
cated that it has decided to make the compliance 
team permanent so that compliance reviews will 
continue at the end of this two-year project. 

In addition, new legislation for developmental 
services has been proposed that includes provisions 
for inspections without a warrant. Further details 
on the criteria under which inspections may be 
conducted without a warrant are to be outlined in 
a regulation, which had yet to be developed at the 
time of this follow-up. Quality assurance mechan- 
isms were under development and may also be 
included in the new regulation and in ministry 
policy directives. 

At the time of our follow-up, the Ministry had 
not yet established the process for taking any neces- 
sary corrective action. 


Serious Occurrence Reporting 


Recommendation 8 
To safeguard more effectively the health and safety 
of individuals living in community accommodations, 
the Ministry of Community and Social Services 
should reassess the objectives of the serious occurrence 
reporting process and, in the light of that reassess- 
ment, it should: 

@ provide agencies with a clear and unambiguous 
definition of the serious occurrences that need to 
be reported; and 

@ design a process that meets its objectives and 
that its regional offices can oversee effectively. 


Status 
The Ministry informed us that it had made the 
following improvements with regard to our recom- 
mendation on serious occurrences: 

e the development of new guidelines and ser- 
vice provider procedures, including clarified 
definitions for serious occurrences; 

e the introduction of a new, streamlined, inte- 
grated reporting form; 

e the identification of best practices for manag- 
ing serious occurrences, and communication 
of this information to regional offices; and 

e more effective use of the automated database. 

Although the Ministry appears to have taken 

several important steps in addressing our rec- 
ommendation, options for reporting physical 
restraints—the most common reason for serious 
occurrences—were still under development at the 
time of this follow-up. Also, a review undertaken 
by the Ministry’s internal audit division noted 
inconsistencies in the regions’ progress in imple- 
menting these improvements: one regional office 
had developed specific business practices for the 
management of reporting procedures for serious 
occurrences to ensure that ministry expectations 
were being met, while another regional office had 
only made changes to the way it used the database. 
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Complaint Procedures 


Recommendation 9 
In order to help ensure that all complaints received 
by agencies get a fair hearing and are satisfactorily 
resolved on a timely basis, the Ministry of Community 
and Social Services should: 

@ requtre all agencies to have a complaints process 
in place that is similar to the process described 
in the Child and Family Services Act and ensure 
that they comply with it; and 

e ensure that all complaints that are escalated to 
a ministry regional office are logged, tracked, 
and resolved fairly and on a timely basis. 


Status 
The Ministry appears to have made little progress 
with regard to implementing our recommendation 
on complaints procedures. It informed us that it 
was attempting to identify best practices for manag- 
ing complaints received by the Ministry and that it 
planned to recommend a set of best practices to the 
regional offices. 

We note that proposed legislation for develop- 
ment services includes provisions that require ser- 
vice agencies to have written procedures to address 
complaints and that the Ministry may provide more 
detail on these requirements in the regulation that 
is to be developed by the end of the 2009/10 fiscal 
year. Complaints procedures for service providers 
may also be required in future policy directives and 
guidelines. 


INFORMATION SYSTEMS 


Recommendation 10 
The Ministry of Community and Social Services 
should ensure that its Service Management Informa- 
tion System (SMIS) provides complete, accurate, and 
useful information on which to base management 
decisions and to help determine whether services pro- 
vided by transfer-payment agencies are effective and 
represent value for money spent. 
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Status 
The Ministry has made some progress towards 
implementing our recommendations regarding 
its information system. It has developed business 
requirements for the automation of the transfer 
payment budget package. Pending development 
of the automated solution, the Ministry has also 
developed a modified budget package and a central 
repository that stores all budget and quarterly data. 
This tool will include the new financial inputs, 


streamlined expenditure categories, service data 
standards, and validation rules to improve the qual- 
ity of data at the point of entry. 

The Ministry has also implemented a web- 
based application, which includes a range of tools 
and analytical reports to assist regional offices in 
improving the completeness and accuracy of SMIS 
data. The completeness, accuracy, and usefulness of 
SMIS reports for management decision-making will 
be evaluated in future audits. 







Section 


Background 


The Ministry of Transportation (Ministry) is respon- 
sible for protecting the public by ensuring that the 
privilege of driving is granted only to persons who 
demonstrate that they are likely to drive safely. 

Although the province does not regulate driv- 
ing schools, the Ministry administers a voluntary 
Beginner Driver Education (BDE) program under 
which driving schools that meet specified require- 
ments can become Ministry-approved course pro- 
viders. They may issue driver-education certificates 
to students who have completed the course success- 
fully; the certificates entitle students to have their 
12-month G1 stage reduced by up to four months 
and, possibly, to save on insurance. Of the approxi- 
mately 218,000 new drivers each year, about 
120,000, or 55%, take the BDE course and about 
67,000 take advantage of the time reduction that 
allows them to attempt their G2 road test earlier. 

In our 2007 Annual Report, we found that 
collision-involvement rates for novice drivers who 
enrolled in the BDE program were significantly 
higher than those for drivers who did not partici- 
pate in the program. Although this statistic is not 
necessarily an indication of the effectiveness of the 
BDE course, the Ministry had not followed up on 


Cha pter 4 Ministry of Transportation 


4A) Driver Education and 
Examination 


Follow-up on VFM Section 3.04, 2007 Annual Report 


the reasons for the higher collision rates. We also 
noted the following: 

@ Our analysis of statistics concerning Ontario 
drivers and a number of other studies showed 
that drivers who had taken advantage of the 
time reduction had higher collision rates than 
those who had remained longer in the super- 
vised stage. 

@ Virtually all the external stakeholders we 
interviewed had expressed concerns about 
the sale of driver-education certificates by 
unscrupulous driving schools to students 
who had not completed the BDE course 
requirements. 

@ The Ministry’s inspection of BDE driving 
schools had not focused on ensuring that the 
training was in accordance with the Ministry- 
approved curriculum. Where inspections had 
been done, they had found many cases of 
repeated non-compliance by driving schools. 

Although the Ministry is ultimately responsible 

for the examination and licensing of drivers, the 
administration of driver examination services is 
outsourced. We noted that there had been signifi- 
cant improvement in the wait times for taking a 
road test, a major issue that also had been noted in 
our last audit in 2001. However, we noted differ- 
ences in the pass rates of examiners that were large 
enough to indicate that applicants had not been 
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passed or failed on a consistent basis throughout 
the province. In addition, there were many cases 
where applicants had not been required to complete 
all necessary driving-test manoeuvres. As well, 
some applicants may have been travelling signifi- 
cant distances from their home area to take their 
test at more distant centres that seemed to have less 
stringent testing requirements. 

The Ministry’s driver-examination outsourcing 
agreement had good oversight mechanisms that 
included a number of performance standards. 
However, in monitoring the performance of the 
service provider, the Ministry found a high number 
of defects that could be indicative of persistent 
problems. 

Under the Driver Certification Program, the 
Ministry designates bodies such as municipalities, 
trucking firms, and school-bus companies as 
Recognized Authorities to operate driver-licence 
training and testing programs for their employ- 
ees. Upon successful completion of the program, 
employees are entitled to have their class G driver 
licences upgraded to a commercial class licence. 
About 8,600 commercial licences, or 20% of such 
licences, are issued this way annually. Ministry 
inspections and investigations of complaints had 
found instances where Recognized Authorities and 
their trainers had been upgrading drivers who had 
neither received any training nor demonstrated the 
necessary driving skills, as well as instances where 
non-employees had been upgraded. 

We made a number of recommendations for 
improvement and received commitments from the 
Ministry that it would take action to address our 
concerns. 


Status of Recommendations 





According to the information we received, the Min- 
istry has made significant progress in addressing 
many of the recommendations in our 2007 Annual 
Report, including improvements to the Ministry’s 


inspection process for the delivery of driver educa- 
tion under the BDE program and stronger controls 
over the issuing of driver-education certificates. 
For a number of our other recommendations, such 
as taking corrective action on differences in pass 
rates and reviewing the appropriate amount of time 
needed to administer road tests, we noted that the 
Ministry’s action was ongoing and there had been 
some progress in implementation. The status of the 
action taken on each of our recommendations at 
the time of our follow-up was as follows. 


DRIVER EDUCATION 
Beginner Driver Education (BDE) Program 


Recommendation 1 
To ensure that novice drivers enrolled in the Beginner 
Driver Education (BDE) program receive effective 
training in safe driving, the Ministry of Transporta- 
tion should evaluate the effectiveness of the BDE 
program, including investigating the reasons for the 
higher collision involvement rates for drivers who 
have completed the BDE program. 


Status 
The Ministry indicated that it had made significant 
changes to improve the BDE curriculum standards 
and strengthen its oversight of driving schools. 
In addition, an internal review of the Graduated 
Licensing System (GLS) had identified several 
improvements to the GLS that would affect the BDE 
program, such as increasing the minimum amount 
of time that a novice driver must remain in a super- 
vised driving stage (G1). The Ministry was in the 
process of seeking the regulatory amendments that 
would allow it to implement these improvements. 

Given this and other significant changes to the 

BDE program described in the section below, the 
Ministry indicated that it would be more appropri- 
ate to wait and evaluate the effectiveness of the 
new BDE program instead of evaluating the pro- 
gram using old standards. It estimated that a com- 
bined evaluation of both the GLS and BDE could be 
done in about three to five years, the time generally 


required for the accumulation of the collision and 
violation data needed for the evaluation of program 
changes. 

The Ministry contracted the Traffic Injury 
Research Foundation (TIRF) to develop a method- 
ology to evaluate the GLS, which is to be used in 
the future to determine the effect of recent changes 
to the system. As part of that contract, TIRF is to 
deliver a safety evaluation and survey of G1 drivers 
who have taken a BDE program and used the time 
discount. Finalized versions of the methodologies 
were expected to be completed by September 2009. 


Driver-education Curriculum, and Drivers with 

Reduced G1 Stage 

Recommendation 2 
To help ensure that new drivers receive adequate 
behind-the-wheel supervised training, the Ministry of 
Transportation should: 

@ update its standards and curriculum to recog- 
nize changes in the driving environment over the 
last decade; and 

e reconsider the desirability of reducing the super- 
vised (G1) driving stage for drivers who success- 
fully complete the Beginner Driver Education 
program. 


Status 

The Ministry informed us that it had updated the 
BDE standards and developed a new curriculum 
that recognized changes in the driving environ- 
ment. The previous standards had focused on the 
mechanics of driving. The new BDE standards 
still include all the mechanics of driving, but add 
a greater focus on driving attitudes, placing par- 
ticular emphasis on risk awareness, attention, and 
responsibility. 

BDE course providers had the option of pur- 
chasing the new curriculum from the Ministry 
or developing their own and submitting it to the 
Ministry for approval by April 17, 2009. Accord- 
ing to the Ministry, all BDE course providers had 
complied and were required by regulation to teach 
the new curriculum as of September 1, 2009. The 
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Ministry will be auditing schools as part of its 
compliance-monitoring to ensure that all of them 
have complied. 

Regarding the desirability of reducing the 
duration of the supervised driving stage for driv- 
ers who have completed the BDE program, the 
proposed changes to the GLS would increase both 
the minimum length of the G1 stage and the time 
BDE drivers must spend in that stage before the time 
discount can apply. Under the existing GLS, novice 
drivers are required to remain in the G1 stage for 12 
months before attempting a road test to progress to 
the unsupervised G2 stage, and this time would be 
reduced to eight months for drivers who success- 
fully complete the BDE program. With the proposed 
changes, the minimum length of the G1 stage would 
be extended to 18 months, and drivers who have 
completed the BDE program would be required to 
remain in a supervised stage (G1) for 12 months. 


Inspection of Driving Schools 
Recommendation 3 
The Ministry of Transportation should ensure that 
driving schools are providing students with training in 
accordance with the Ministry-approved curriculum by: 
e developing and following a more comprehensive 
and curriculum-based approach to inspection; 
e following up on deficiencies found and taking 
more definitive action where repeat violations 
are being noted; and 
e working with the Ministry of Government 
Services to help inform students about driving 
schools that are on its Consumer Beware List. 


Status 
The Ministry indicated that it had implemented a 
new audit program in December 2008 that involved 
the inspection of all schools, with a special focus 
on those considered high-risk. The audit program 
takes into account previous audit results and 
includes features such as an improved field-audit 
program for opening new schools, compliance with 
Ministry standards, fraud prevention, and aware- 
ness of identity theft (which occurs when students’ 
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information is falsely used to book road tests). The 
new process also placed more emphasis on inspect- 
ors’ attending sessions to observe what is being 
taught in a class. As well, the Ministry indicated 

it had continued to conduct “mystery shopping,” 
where people engaged by the Ministry pose as 
members of the public and enroll in a BDE course, 
then evaluate whether the course was delivered in 
accordance with ministry standards. 

The Ministry also stated that it had taken action 
to revoke the “Ministry-approved” status from 
schools that were found to have committed fraud. 
After audits conducted in 2007, eight schools had 
their status revoked for selling driver-education 
certificates and another 23 schools for teaching 
below standards. Depending on the severity of the 
non-compliance, the Ministry also initiated follow- 
up audits to ensure that any compliance issues had 
been rectified. According to the Ministry, its action 
to date had been effective and subsequent audits 
had indicated that, overall, driving schools were 
complying with Ministry standards. 

Regarding driving schools that were on the Min- 
istry of Consumer Services (previously the Ministry 
of Government Services) “Consumer Beware List,” 
the Ministry has been posting information on its 
website for students considering or currently taking 
BDE, to make them aware of program requirements 
and the responsibilities of the BDE course provider. 


Driver-education Certificates 

Recommendation 4 
The Ministry of Transportation should strengthen 
its controls to minimize the risk of driver-education 
certificates being issued to students who have not 
completed the required driver training. It should also 
follow up on any suspicions of fraudulent selling of 
certificates and take immediate action where such 
suspicions are confirmed. 


Status 
The Ministry indicated that, beginning in Septem- 
ber 2008, it had replaced paper driver-education 
certificates with an electronic interface that links 


driver-education certificate data to the driver-test 
database. This eliminates the need to issue and 
keep track of blank paper certificates and prevents 
schools that are not Ministry-approved from gain- 
ing access to certificates. The electronic system 
matches the name and driver licence number with 
that in the driver-test database to ensure that the 
records and identity are legitimate. The system also 
does not allow back-dating of course completion 
information and a student can only receive a time 
discount at the time that they book their test. These 
checks help to prevent unscrupulous providers from 
readily selling driver-education certificates. 

The Ministry also informed us that only certified 
instructors and schools have access to the system, 
and the volume of certificates issued by schools was 
being tracked and monitored. Any school found (for 
example, through an audit) to be graduating stu- 
dents who had not completed a BDE course would 
have its Ministry-approved status revoked and its 
access to the system cut off. The Ministry can also 
lay charges against a school under the Highway 
Traffic Act, if warranted. 


Non-Ministry-approved Driving Schools 


Recommendation 5 
To protect the public, the Ministry of Transporta- 
tion should work with the Ministry of Government 
Services and take action to ensure that only legitimate 
course providers are allowed to operate and advertise 
as Ministry-approved course providers. 


Status 
As of June 2007, it is an offense under the Highway 
Traffic Act for a school to advertise itself as Ministry- 
approved if it is not. In the past, the Ministry’s 
response to any non-approved school engaged in 
false advertising of this sort was to issue a cease- 
and-desist order. The Ministry informed us that 
it had since strengthened its response and would 
lay charges on any school making a false claim, if 
warranted. Regulations implemented in September 
2007 have also helped to accelerate the timeframe 


for sanctioning a school for failing to meet the BDE 
standards, and to help prevent sanctioned owners 
from reopening a new school under a different 
name. 

Effective December 2007, only those driving 
instructors working in Ministry-approved schools 
providing BDE courses were allowed to teach new 
drivers in Ontario. The Ministry indicated that 
the change from paper-based driver-education 
certificates to an electronic system would prevent 
non-approved schools from having access to driver- 
education certificates and anyone attending these 
schools would not receive a certificate. 


Driving Instructors 


Recommendation 6 
To ensure that student drivers receive proper training, 
and to protect the safety of the public, the Ministry of 
Transportation should: 

e update the driving-instructor curriculum and 
consider reinstituting training for new master 
driving instructors; 

@ consider strengthening the training require- 
ments for maintaining a driving-instructor 
licence; and 

e@ ensure that instructors who are the subject of 
numerous complaints are more stringently dealt 
with. 


Status 
The Ministry informed us that it had developed 
standards for training new master driving instruct- 
ors, but a curriculum based on the new standards 
had yet to be developed. The Ministry informed us 
that it was to meet with industry stakeholders in 
fall 2009 to work on designing new curricula for 
master driving instructors and driving instructors. 
The Ministry also indicated that it had taken 
action to strengthen the requirements for obtaining 
and maintaining a driving instructor licence. The 
new requirements included no demerit points and 
criminal convictions to be allowed before becom- 
ing licensed, licences to be revoked for any fraud 
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conviction, and a reduction in allowable demerit 
points after licensing. The Ministry had reviewed all 
existing driving instructor licences according to the 
revised policy and it resulted in the revocation of 
over 150 instructor licences. The Ministry was also 
in the process of implementing a requirement that 
all driving instructors take refresher courses within 
a specified time period. 

Regarding driving instructors who had been 
the subject of numerous complaints, the Ministry 
informed us that it could assess instructors on “fit 
and proper” grounds if they engaged in fraudulent 
activities, such as selling driver-education certifi- 
cates. This was a new approach—the Ministry was 
previously unable to take such actions. Revoking an 
instructor’s licence under the fit and proper clause, 
although a complicated process requiring a great 
deal of information, at least exists as an option 
where it did not before. 


Driver Certification Program 


Recommendation 7 
To minimize risk to the safety of the public and given 
the concerns that are arising from current inspections 
of those organizations that are allowed to train and 
test drivers for the more advanced licence classes, the 
Ministry of Transportation should: 

@ comply with its policy to inspect those organiza- 
tions annually and expand its inspection to 
include the training and examination processes; 
and 

@ pay particular attention to the risk of those 
organizations providing an advanced class of 
licence to unqualified drivers. 


Status 
The Ministry indicated that, to improve program 
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integrity and strengthen consumer protection, 

it had developed a risk-based inspection process 

to target and follow up on high-risk Recognized 
Authorities every one to three years. The new 
process included inspection of training and 
examination processes. Should an audit reveal non- 
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compliance with the program, an administrative 
suspension would be applied against the Recog- 
nized Authority. As of March 31, 2009, the Ministry 
had completed audits of all Recognized Authorities 
not inspected in the last two years. According to 
the Ministry, any organization that had a previous 
history of providing advanced-class licences to 
unqualified drivers would be automatically desig- 
nated high-risk. 


DRIVER EXAMINATIONS 


Monitoring the Service Provider's Delivery 
of Examination Services 


Recommendation 8 
To help ensure that the outsourced driver-examina- 
tion function meets its objective of passing only quali- 
fied persons, the Ministry of Transportation should: 

@ conduct compliance monitoring according to 
the frequency established under its risk-based 
approach; 

@ provide the service provider with more informa- 
tion on systemic non-compliance areas noted 
where a formal default notice was not issued 
and ensure that such areas are specifically 
assessed in future compliance reviews; and 

@ enhance the query and reporting capabilities of 
the management information system to enable a 
more proactive approach to identifying the more 
serious and recurring problems. 


Status 

The Ministry informed us that its compliance- 
monitoring focus had shifted to monitoring high- 
risk and problem locations. It developed new rating 
standards based on the history of performance 
and past defaults, geographic location, transaction 
volume, etc., to help assess the frequency of compli- 
ance monitoring. These standards have been dis- 
tributed to all field staff to ensure uniform practice 
and compliance with standards. 

Ministry staff advise Drive Test Centres of any 
performance problems immediately following 
an audit. In addition, new governance structures 


were put in place in fall 2007 to better collaborate 
and share information with the service provider. 
Working committees, made up of staff from both 
the Ministry and service provider, have also been 
meeting regularly to discuss various operational 
and compliance issues and make recommendations 
on specific issues. 

The Ministry informed us that it had remedied 
the problems with its compliance-monitoring 
reporting system. The system is able to supply the 
service provider with monthly data and reports 
on systemic non-compliance areas, organized by 
location and performance standards. In addition to 
normal reporting, the system can run ad hoc analy- 
ses more easily than it could in the past. Previously, 
any non-standard reporting would have required a 
significant amount of programming effort. 


Results of Driver Examinations 


Recommendation 9 
To ensure that driving examinations are carried out 
consistently across the province, the Ministry of Trans- 
portation should: 
® investigate significant differences in the pass 
rates of individual examination centres and 
require corrective action to reduce the differ- 
ences; and 
@ review the time needed to administer road tests 
with all required manoeuvres being tested, 
recognizing that this may necessitate either less 
or more time depending on the municipality in 
which the centre is located. 


Status 
The Ministry indicated that its service provider 
had formalized procedures and protocols for Drive 
Test Centre supervisors to use when examiner pass 
rates deviate from the norm. It was establishing 
an acceptable pass-rate norm for each Drive Test 
Centre and examiners are to be monitored against 
this acceptable norm. 

The Ministry and the service provider initiated a 

comparison of the characteristics of all routes used 


with the requirements prescribed by road tests. 
The purpose of this exercise was to identify gaps 
that might prevent requirements and manoeuvres 
from being met as a result of the required road 
infrastructure being absent from a test location, 

as well as to estimate the time needed to reason- 
ably complete a test using that route. The Ministry 
expected all routes to have been driven by the end 
of September 2009 and all information to have 
been consolidated by October 2009. 


Customer Service 


Recommendation 10 
To maintain a high level of customer service, the 
Ministry of Transportation should periodically 
monitor the service provider’s compliance with its 
customer-service performance standards, including its 
complaint-handling and -resolution process. 


Status 
The Ministry informed us that a customer survey 
had been conducted at all Drive Test Centres in fall 
2007 and the final results presented to the Ministry 
in August 2008. Overall, province-wide satisfaction 
was reported to be 87%. The Ministry indicated that 
it had continued to conduct annual customer sur- 
veys and work with the service provider to develop 
action plans to address annual survey results. 

In addition, the Ministry worked with its service 
provider to develop a Customer Care Report proto- 
col. The initial report, for the period May through 
July 2008, was finalized in October 2008. The Min- 
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istry was to continue to produce the Customer Care 
Report semi-annually and was to work with the 
service provider to develop action plans to address 
the Report’s findings. 


Performance and Training of Examiners 


Recommendation 11 
To maintain a high standard for driving examina- 
tions, the Ministry of Transportation should ensure 
that: 
@ All driver examiners receive the required train- 
ing; and 
@ Their work is evaluated periodically and effect- 


ive performance management procedures are 
followed. 


Status 
The Ministry informed us that the service provider 
had provided comprehensive records for all train- 
ers, driver examiners, and customer service agents. 
This assured the Ministry that the required training 
had been given and that records had been properly 
documented and maintained. 

In addition, since our 2007 audit, the service 
provider has provided the Ministry with semi- 
annual reports on in-car examinations to prove 
that they were done and to ensure that examiners’ 
qualifications were up to date. Spot-auditing was 
also conducted as part of compliance-monitoring 
activities to ensure that only qualified examiners 
were conducting only those road tests for which 
they were trained. 
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Section 


Background 


Ontario has a number of drug programs that pro- 
vide prescription drugs to Ontario seniors, social 
assistance recipients supported by the Ontario 
Works and Ontario Disability Support programs, 
and certain other types of eligible recipients as 
defined under legislation. The Ministry of Health 
and Long-Term Care (Ministry), through the Health 
Network System (Network), processes payment 

of drug claims, and provides on-line information 

to pharmacists. The Network is linked to approxi- 
mately 3,050 pharmacies and 100 other dispensers. 

The Network processes 109 million prescrip- 
tions annually for approximately 3.2 million 
eligible recipients. For the 2008/09 fiscal year, total 
expenditures for Ontario’s drug programs were 
$4.1 billion ($3.7 billion in 2006/07), of which 
$774 million($742 million in 2006/07) was paid by 
the Ministry of Community and Social Services for 
drug benefits for social assistance recipients. 

Our 2007 audit indicated that the externally 
managed Network generally processed drug claims 
in accordance with legislative requirements and 
ministry policy. However, to further control costs, 
the Ministry must be more vigilant in ensuring that 
the risks related to ineligible claimants and unusual 
drug claim patterns are appropriately addressed. 
Specifically: 
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e The Ministry did not closely monitor phar- 


macists’ use of system override codes to 

grant drug coverage eligibility to recipients 
identified by the system as ineligible for drug 
coverage. System overrides must be sup- 
ported by appropriate documentation such as 
temporary eligibility cards. For instance, one 
pharmacy made more than 300 claims in a 
five-month period through system overrides 
for one individual who was ineligible for drug 
coverage during that time. 

When pharmacists acquire drugs at costs 
greater than the Ontario Drug Formulary 
(Formulary) prices, they can be paid at these 
higher drug prices by entering a price override 
code in the system. Our review of a sample of 
price override claims paid by the Ministry in 
February 2007 found that more than 30% of 
the unit drug prices in these claims exceeded 
their Formulary prices by more than 100%. 

In one case, the price claimed exceeded the 
Formulary price by 12,500%, resulting in the 
Ministry paying almost $2,400 for a claim 
that, according to the Formulary price, should 
have cost less than $20. 

Our previous audits in 1996 and 2001 and 
the Standing Committee on Public Accounts 
all expressed concern about the decline in 
field inspections of pharmacies. Our work in 
2007 indicated that, at the Ministry’s current 
inspection rate of 3% of dispensing agencies 
in a year, it will take up to 30 years to cover all 


agencies. The Ministry needs to target high- 
risk dispensing agencies identified through 
activities such as a review of unusual claims 
statistics. For instance, our review of 2005/06 
claims data found that 20 dispensing agencies 
filled prescriptions for an average drug supply 
of less than three days, which allowed them 
to charge more dispensing fees, yet only one 
of these agencies was inspected in the last 

six years. In conjunction with the Ministry, 
we selected a dispensing agency that hada 
high number of claims per drug recipient and 
attended the related field inspection. This 
single inspection identified $270,000 in over- 
payments, of which $240,000 was because of 
claims for invalid dispensing fees. 

We made a number of recommendations for 
improvement and received commitments from the 
Ministry that it would take action to address our 
concerns. 


Status of Recommendations 





The Ministry has made good progress in addressing 
most of our recommendations, although several 
will take another year or two to be fully addressed. 
The status of the actions taken on each recommen- 
dation at the time of our follow-up is as follows. 


ELIGIBILITY FOR DRUG COVERAGE 


Recommendation 1 
To ensure that only eligible recipients receive or con- 
tinue to receive drug coverage, the Ministry of Health 
and Long-Term Care (Ministry) should ensure that: 
@ income levels of seniors receiving reduced 
co-payments are supported by proper docu- 
mentation or through electronic means, such as 
the Canada Revenue Agency income link; 
@ eligibility override codes used by pharmacists 
are applied and supported appropriately; 
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@ the use of override codes is monitored and 
abnormally high override rates are investigated; 
and 

® continuing eligibility of long-term-care residents 
is confirmed independently by obtaining infor- 
mation from the long-term-care homes or the 
Ministry’s Long-Term Care Program. 


Status 
The Ministry advised us that the electronic link to 
Canada Revenue Agency (CRA) income information 
for the Trillium Drug Program is in place; the Min- 
istry was in the process of enhancing this link at the 
time of our follow-up and expected it to be complete 
by the end of the 2009/10 fiscal year. Once enhance- 
ment is completed, the Ministry will determine what 
systems and program-administration changes are 
required to implement electronic income verifica- 
tion through the CRA for the Seniors’ Co-payment 
Program. Seniors applying for the Seniors’ Co-pay- 
ment Program are to provide proof of income and 
a signed consent form, and the Ministry is to verify 
their eligibility for benefits using the CRA link when 
this feature becomes available. 

The Ministry informed us that because the use 
of override codes was specific to the situation and 
the clinical decision of the pharmacist who initiated 
the code, it did not set benchmarks for the use of 
these codes. Instead, the Ministry’s inspection staff 
had been ensuring compliance by reviewing claims 
and monitoring any abnormal activity relating 
to the use of an override code, and investigating 
where appropriate. In their reviews, inspection 
staff had been considering the clinical expertise of 
pharmacists who had decided to use the code and 
requesting documentation to support any claims 
submission. 

Because there is no data repository of long- 
term-care residents in the Ministry’s Long-Term 
Care Program, the Ministry had not been able to 
use internal information to verify the continuing 
eligibility of these residents. To support its audit 
function, the Ministry had doubled its inspection 
resources for doing site-inspections and auditing 
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claims submitted by pharmacies. Where applicable, 
the Ministry would check with long-term-care 
homes to verify the eligibility of residents. 


PROCESSING OF PAYMENT CLAIMS 
Electronic Processing of Payment Claims 


Recommendation 2 
To help ensure that all claims are processed accurately 
and completely in accordance with legislative and 
policy requirements, the Ministry of Health and Long- 
Term Care should: 

@ periodically perform Health Network System 
(Network) assessments or tests to identify areas 
of non-compliance, with particular emphasis on 
ensuring that the Network has been updated for 
program changes; and 

e regularly obtain information from the Ontario 
College of Pharmacists (College) on pharmacy 
closings to update the Network database. 


Status 
The Ministry cited the following initiatives it had 
put in place to identify areas requiring updates to 
the Network since our 2007 audit: 

@ In 2008, the Ministry instituted a governance 
process for information technology changes 
to the drug program portfolio. Specifically, a 
team of senior managers from the business, 
policy, and information technology areas 
relating to drug programs were meeting 
monthly to prioritize projects and to monitor 
the status of projects under way and assess the 
impact of any proposed policy changes on the 
Network. 

@ In 2008, the Ministry terminated all prior 
agreements and entered into a new agree- 
ment with all the dispensing agencies in the 
Network. This new agreement reinforced 
to dispensing agencies that claims must be 
submitted in compliance with the Ministry’s 
legislative and policy requirements and pro- 
vides the Ministry with clearer authority to 
terminate agreements. 


e The Ministry completed a review of adjudica- 
tion rules for recipients under the Ontario 
Works drug program. As a result of this 
review, the Ministry updated the information 
in the Network in February 2009 to allow a 
maximum 35-day supply of drugs for recipi- 
ents in the Ontario Works program even if 
they are also registered in the Trillium Drug 
Program (some recipients had been receiving 
up to a 100-day supply of medication). 

e In 2008, subsequent to a review of claims data 
on the frequency of dispensing in the Ontario 
Drug Benefit Program, legislative changes 
were made such that the Ministry would pay a 
maximum of two dispensing fees to dispensers 
for the supply of a listed drug product in a cal- 
endar month. The Ministry believes that these 
changes will result in substantial savings. 

The Ministry informed us that the Ontario 
College of Pharmacists regularly notified it of new 
pharmacy openings and changes of ownership. In 
addition, the Ministry told us that it had updated 
the Network and reconciled pharmacy listings in 
2008 and early 2009. 


Cost-to-operator Payments 


Recommendation 3 
To ensure that it pays drug prices charged in excess 
of Formulary prices only when appropriate, the Min- 
istry of Health and Long-Term Care should: 

e regularly review and monitor pharmacy claims 
for manufacturer costs exceeding Formulary 
prices for accuracy and for evidence of manufac- 
turer invoice support; and 

@ take appropriate action to recover overpay- 
ments when claims are found to be invalid or 
incorrect and when drug manufacturers are in 
non-compliance with Formulary prices. 


Status 
The Ministry informed us that it had reviewed 
all Formulary products to ensure that informa- 
tion was current and accurate and that some 300 


discontinued products had been removed. The 
Ministry estimates that this will result in savings 
that could average several million dollars annually. 
The Ministry had also recovered overpayments of 
$2.3 million between January 2008 and June 2009, 
an 18-month period, which it found during routine 
audits that identified discrepancies in claims. As of 
June 2007, the Ministry no longer allows the use 
of cost-to-operator intervention codes for almost — 
all generic drugs, as well as for brand-name drugs 
with price agreements and no equivalent inter- 
changeable products (at the time of this follow-up, 
approximately 60% of brand-name drugs fell into 
this latter category). Blocking the use of such codes 
prevents pharmacies from charging more than the 
drug-benefit price that is listed in the Formulary. 
The Ministry informed us that it was allowing 
the use of intervention codes for brand-name 
drugs that are without price agreements but that 
legislation requires that a manufacturer supplying 
a listed drug product for the purposes of the public 
drug program must sell it at the drug-benefit price 
listed in the Formulary. Regarding claims for single- 
source drugs, which are typically brand-name prod- 
ucts, the Ministry informed us that the Network 
was not paying dispensers an amount higher than 
the drug-benefit price. 


Manual Processing of Paper Claims and 
Reimbursement Receipts 


Recommendation 4 
To ensure that all manual claims are valid and are 
accurately processed in compliance with legislative 
and policy requirements, the Ministry of Health and 
Long-Term Care should conduct regular quality- 
assurance reviews of such claims. 


Status 
The Ministry informed us that it had implemented 
a quality-assurance process in April 2007, and 
had since made adjustments to, and continued to 
review, this process. As part of its quality-assurance 
process, the Ministry was completing random 
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checks of approximately 10% of claims on a weekly 
basis. The Ministry was correcting errors identi- 
fied through these checks and using them in its 
refresher training where possible. 


INDIVIDUAL CLINICAL REVIEWS 


Recommendation 5 
To more effectively identify high-request drugs for 
inclusion in the Formulary, the Ministry of Health 
and Long-Term Care should consider tracking Individ- 
ual Clinical Review drug approvals by diagnosis type 
and the related numbers of requests. 


Status 
The Ministry advised us that the existing Individual 
Clinical Review (ICR) system provided information, 
mainly volume and cost data, to support decisions 
on whether or not a drug should be included in the 
Formulary. However, the Ministry informed us that 
ICR was to be replaced by a new program that will 
capture additional information. 


TRILLIUM DRUG PROGRAM 


Recommendation 6 
To ensure that the Trillium Drug Program is adminis- 
tered in accordance with legislative requirements, the 
Ministry of Health and Long-Term Care should: 
e ensure that households provide appropriate 
documentation verifying income; and 
e develop and implement appropriate policies and 
procedures to pursue unpaid deductibles owed 
to the Ministry. 


Status 
The Ministry informed us that it had implemented 
monthly inspections of randomly selected house- 
hold files to verify that Trillium applications were 
processed with the required income documenta- 
tion. Whenever the Ministry encountered missing 
or incomplete documentation, it sent a letter asking 
the applicant to submit the required information. 
Similarly, the Ministry was issuing termination let- 
ters to households that had not accessed program 
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benefits in over 18 months, as required by ministry 
policy. 

To reduce the amount of unpaid deductibles 
in the future, the Ministry implemented policies 
limiting the quantities of drugs to be dispensed in 
the third and fourth quarters of a benefit year prior 
to all deductibles being paid. The Ministry also 
informed us that it had conducted an analysis in 
July 2008 to better understand the potential costs 
of recovering unpaid deductibles. It concluded 
on the basis of this analysis that collecting unpaid 
deductibles would not be cost-effective. 


SPECIAL DRUGS PROGRAM 


Recommendation 7 
To ensure that the cost of special drugs used is mint- 
mized, the Ministry of Health and Long-Term Care 
should: 

@ develop and implement appropriate and consist- 
ent policies and procedures relating to the Spe- 
cial Drugs Program that address the supporting 
claim information required, including details 
about drug quantities and unit prices paid, and 
the acceptable reimbursement method; 

@ consider securing more contracts with drug 
companies for better special-drug prices; and 

@ consider conducting periodic reviews of hospital 
supporting records to verify the accuracy and 
validity of the amounts claimed. 


Status 
The Ministry advised us that it had reviewed the 
operational procedures for reimbursement of 
drug costs under the Special Drugs Program. At 
the time of our follow-up, the Ministry was asking 
hospitals submitting claims for amounts greater 
than the prices allowed by the Formulary to submit 
documentation with the claim. In the Special Drugs 
Program specifically, where the Ministry has stand- 
ing agreements with two drug manufacturers that 
they will not charge the hospitals higher costs than 
the agreed-upon prices, the Ministry advised us 
that detailed information on drug quantities and 


unit prices paid was not always available from cer- 
tain hospitals because they dispensed bulk supplies 
for patients (for instance, through clinics). The 
Ministry also informed us that it had started tran- 
sitioning the method used to reimburse hospitals’ 
claims. The new method would reimburse hospitals 
on the basis of actual drug use rather than on the 
amount of special drugs purchased. However, some 
hospitals were not able to apply the new method 
because administrative records of patients’ medica- 
tion were not always computerized. This was the 
case, for example, in hospital clinics where patient 
dispensing (medical administration) records were 
not kept in electronic format, and actual usage 
information was not readily available. 

With respect to securing more contracts with 
drug companies for special drugs at better prices, 
the Ministry informed us that it had reviewed pro- 
gram expenditures and secured contracts for high- 
cost drug products and their generic equivalents. 
The Ministry will monitor low-volume and low-cost 
products for potential agreements should costs 
increase. 

The Ministry informed us that it had conducted 
reviews on all invoices submitted for reimburse- 
ment to identify any excessive unit prices and 
manufacturer’s discounts to be deducted from 
its payments to hospitals. The Ministry was also 
reviewing reports on dispensing and usage to con- 
firm the amounts claimed. 


INSPECTION AND VERIFICATION 


Recommendation 8 
To promote thorough and effective inspections that 
encourage ongoing compliance, the Ministry of Health 
and Long-Term Care should: 

@® conduct a review of the inspection staffing 
resources and develop an overall audit plan to 
ensure that sufficient inspection resources are in 
place to provide adequate inspection coverage 
across the province; 


@ onaregular and systematic basis, select dispens- 
ing agencies for inspection using appropriate 
risk factors; 

@ provide inspectors with ongoing formal audit 
training in how to conduct an audit, including 
risk assessment, development of inspection pro- 
grams, file completion and documentation, and 

follow-up requirements; and 

@ deter repeat offenders by enforcing existing 
legislative penalties. 


Status 
At the time of our 2007 audit, the Ministry had 
three full-time inspection field staff. At the time of 
our follow-up, the Ministry had hired another three 
full-time inspectors who had trained as pharma- 
cists, and planned to hire two pharmacy technicians 
to support routine desk audits. In addition, the Min- 
istry had negotiated an agreement with the Min- 
istry of Revenue to assist with auditing the Ontario 
Drug Benefit Program in the 2009/10 fiscal year. 

The Ministry also advised us that it had 
developed a training manual that helps it to identify 
risk and carry out its inspections of agencies. In 
addition, the Ministry told us it had developed a 
standardized auditing process to assist new staff 
and promote consistency and accountability in its 
inspection procedures. 

In our 2007 Annual Report, we noted that 
ministry policy allowed for interest penalties to be 
charged when recovery of overpayments is repaid 
by instalments, and the Ministry could take court 
action to penalize dispensing agencies for offences 
identified under the Ontario Drug Benefit Act. 
However, the Ministry did neither at that time. The 
Ministry informed us that provincial offence char- 
ges under the Ontario Drug Benefit Act were laid for 
the first time in April 2009. 


CONTRACT MANAGEMENT 


Recommendation 9 
To ensure that the third-party processor of the Trillium 
Drug Program and the Seniors Reduced Co-payment 


Drug Programs Activity 


Program complies with the terms of its contract, the 
Ministry of Health and Long-Term Care should: 

@ regularly conduct ongoing audits of the third- 
party processor’s records and supporting docu- 
ments to confirm the accuracy and validity of 
the amounts invoiced; and 

@ develop and implement the necessary ministry 
information reports to facilitate reconciliation 
of the amounts invoiced. 


Status 
The Ministry informed us that a dedicated staff 
member was supposed to perform ongoing inspec- 
tion (a “walk-around”) and verification of the 
third-party processor’s records on a monthly basis; 
however, due to increased workload, the staff mem- 
ber did not perform these inspections regularly in 
most of 2009, but resumed this responsibility in 
August 2009. In addition, the Ministry had planned 
to conduct more complete site visits to the third- 
party processor on an annual basis, and had done 
so in October 2007 and May 2009. These site visits 
included an evaluation of the following areas: 

® security clearances for staff; 

safeguarding of personal information; 
inventory of assets; 
training provided to staff; 


quality control of applications, re-assessments, 
renewals, and receipts; and 

® customer calls. 

The Ministry of Finance, as part of its Financial 
Assurance Program, also visited the site of the 
third-party processor and reviewed its transaction- 
processing systems and controls in March 2009. 

The Ministry told us that it had not yet finished 
developing the necessary internal information 
reports to reconcile the amounts invoiced by the 
third-party processor. The Ministry expected to 
have some reports for the Trillium Drug Program 
by the 2009/10 fiscal year and a new client-server 
application for the Trillium Drug Program and the 
Seniors’ Co-payment Program by April 2011. 


Kye! 
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PERFORMANCE MANAGEMENT 


Recommendation 10 
To better monitor and assess the performance of the 
Ministry of Health and Long-Term Care in meeting its 
objectives, the Ministry should: 

e regularly measure and report actual results 
against the performance standards, with vari- 
ances, if any, being resolved on a timely basis; 

@ comply with its correspondence standards in 
handling complaints and take corrective action 
when response times exceed ministry standards; 
and 

e track and analyze the types of complaints and 
inquiries received about pharmacy practices in 
order to identify areas for corrective action or 
improvement. 


Status 
In our 2007 Annual Report, we noted that the 
Ministry had no performance standards for work 
conducted internally to monitor quality of services 
and post-payment verification (for example, inspec- 
tion workload standards). The Ministry indicated 
to us that it had since established indicators for 
evaluating performance for these activities, includ- 
ing a target number of audits and inspections to 
be performed by each inspector. The Ministry was 


continuing to monitor inspection workloads and 
expected that it would be conducting reviews on a 
bi-annual basis. 

In our 2007 Annual Report, we noted that the 
Ministry was exceeding its standard for responding 
to complaints and inquiries by an average of 11 
days. The Ministry told us that it had since met the 
standard it had set for correspondence turnaround 
time—rush requests were processed within 72 
hours, biologic requests within 15 business days, 
and non-rush requests within eight weeks—and 
that management was monitoring the progress of 
all requests and reducing backlog by reallocating 
staffing resources where necessary. The Ministry 
also developed and implemented a joint tracking 
system to keep track of drugs-related correspond- 
ence and issues and ensure timely turnaround in 
two program areas. 

With respect to complaints and inquiries 
received about pharmacy practices, the Ministry 
informed us that it was treating such complaints 
and inquiries in the same way it had other cor- 
respondence: dedicated ministry staff track, file, 
account for, and respond to such complaints and 
inquiries. The Ministry also stated that it was refer- 
ring complaints and inquiries on pharmacy practi- 
ces to the Ontario College of Pharmacists. 






Section 


Background 


The Ministry of Natural Resources (Ministry) 
promotes the sustainable use and development 

of Ontario’s natural resources so that nature can 
renew itself and be available for the use and enjoy- 
ment of future generations. The Ministry estimates 
that each year, 6.7 million Ontarians enjoy recrea- 
tional fishing, hunting, and wildlife viewing, and 
that these activities are worth nearly $6.3 billion 
annually to the provincial economy. Funding for 
the Ministry’s Fish and Wildlife Program for the 
2008/09 fiscal year was $95.8 million ($74.2 mil- 
lion in 2006/07). 

In our 2007 Annual Report, we concluded that 
although the Ministry gathered data and carried 
out assessments on fish and wildlife resources, 
the information was neither sufficient nor current 
enough to provide assurance that the Ministry was 
effectively managing resources for sustainability. 
Although the Ministry had taken some steps to 
address biodiversity and sustainability issues with 
a formal biodiversity strategy and the enactment 
of the Endangered Species Act, 2007, we noted a 
number of plant, fish, and wildlife species whose 
sustainability was of increasing concern. Our obser- 
vations included the following: 

e Although the Ministry had issued guidelines 

and frameworks to protect fish and wildlife 
habitats, it had no comprehensive inventory 
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of the critical habitats key to the recovery or 
sustainability of fish and wildlife resources. 
Identifying these habitats would help the Min- 
istry develop strategies to protect them from 
further degradation. 

In 2002, we recommended that the Ministry 
develop an overall strategy for the conserva- 
tion, protection, restoration, and propagation 
of species at risk. In 2007, we noted that such 
a strategy had subsequently been drafted but 
had not yet been approved or implemented. 
Of the 120 recovery strategies in various 
stages of development for endangered and 
threatened species, only two of the approved 
strategies related to species deemed most 

at risk. For example, the golden eagle had 
declined to six nesting pairs in Ontario, yet no 
recovery strategy was in place. 

The Ministry lacked complete and current 
data on moose populations, which led to the 
issuing of more hunting tags than recom- 
mended by harvest guidelines. For example, 
41 of the 66 geographic areas that calculate 
moose-harvest quotas and allocate hunting 
tags reported a huntable population greater 
than the estimated total population. 


e Although the Ministry had developed a draft 


recovery strategy for the forest-dwelling 
woodland caribou, a threatened species in 
Ontario, it had been slow to finalize and 
implement it. 
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e@ The Ministry’s management of commercial 
fisheries had been largely successful in pro- 
moting the sustainability of commercial 
fish stocks, but there was a need for better 
monitoring and enforcement. In a number of 
cases, commercial and aboriginal operators on 
Lake Superior and Lake Huron exceeded their 
catch quotas by more than 200%. 

e Areduction in deterrent patrols by conserva- 
tion officers and gaps in enforcement coverage 
may have put added pressure on the prov- 
ince’s fish and wildlife resources. For example, 
we noted that when a two-week enforcement 
blitz was carried out in 2006, officers seized 
57 moose that had been hunted illegally— 
almost double the 29 animals seized during a 
similar blitz in 2005. 


Status of Recommendations 





According to information received from the Min- 
istry, we noted that some action has been taken 
on all of the recommendations in our 2007 Annual 
Report. The Ministry has made policy changes, 
launched a number of initiatives, and committed 
significant additional funding to address most of 
these recommendations. In a number of areas, 
several years may be required to fully address the 
recommendations. The status of action taken on 
each of our recommendations is as follows. 


BIODIVERSITY 
Ontario’s Biodiversity Strategy 


Recommendation 1 
To better ensure that Ontario can meet its commit- 
ments under the Canadian Biodiversity Strategy, 
which was adopted by the province in 1996, the Min- 
istry of Natural Resources should: 


e develop a comprehensive plan for implementing 
its biodiversity strategy, along with appropriate 
time frames; 

e review the adequacy of resources devoted to 
biodiversity; 

e clearly define biodiversity outcomes and per- 
formance indicators to measure progress; and 

@ prepare a comprehensive report on the overall 
state of biodiversity in the province. 


Status 
The Ministry informed us that although it had not 
developed a comprehensive plan for implementing 
its biodiversity strategy, a biodiversity implementa- 
tion team was established to address the 37 actions 
in the Ontario Biodiversity Strategy. This will help 
the Ministry achieve the overall goals and object- 
ives of the strategy. The Ministry allocated an addi- 
tional $850,000 during the 2008/09 fiscal year to 
support the development of biodiversity reporting 
activities and to hire project staff. 

The Ministry has completed a draft Biodiversity 
Outcomes Framework to measure progress towards 
meeting its commitments under the Canadian 
Biodiversity Strategy. The Framework will define 
the strategic outcomes for biodiversity and how 
the Ministry will work to meet its commitment. 
Pending approval, the Ministry plans to implement 
the Framework in the 2009/10 fiscal year. In May 
2008, the Ontario Biodiversity Council released an 
interim report on Ontario’s biodiversity as a first 
step toward preparing a comprehensive report on 
the state of Ontario’s biodiversity. The Ministry 
plans to release a comprehensive report in 2010 
to meet its commitment under the Canadian Bio- 
diversity Strategy. 


Habitat Protection 


Recommendation 2 
To help protect fish and wildlife habitats from further 
loss, alteration, and fragmentation and to preserve 
biodiversity, the Ministry of Natural Resources 
should identify the key habitats that are critical to 


the continued sustainability of native species and pre- 
pare timelines for the development of management 
plans to protect those habitats. 


Status 
The Ministry informed us that habitat protection 
and conservation is being achieved through several 
activities, including management plans for parks 
and protected areas, input into municipal official 
plans, implementation of policies and legislation, 
and input into resource management planning. 
These actions are being accomplished through the 
Endangered Species Act, 2007, through manage- 
ment plans for forestry and fisheries, and through 
management frameworks for key wildlife species. 
In addition, the Ministry is developing a legislative 
regulation that will define the areas of habitat to 
be protected for nine of the top 10 endangered and 
threatened species. The habitat protection regula- 
tions for all nine species were expected to be in 
place by the end of 2009. 


Species at Risk of Extinction in Ontario 


Recommendation 3 
To more proactively manage species at risk and help 
sustain and increase endangered populations, the 
Ministry of Natural Resources should: 
e finalize and put into place its Species at Risk 
Strategy for Ontario; and 
@ prepare and implement a recovery plan with 
related time frames for necessary actions for 
each of the species listed in Ontario as endan- 
gered or threatened. 


Status 
The Ministry informed us that its Species at Risk 
Strategy for Ontario has been finalized and incor- 
porated into Ontario’s Biodiversity Strategy. The 
Endangered Species Act, 2007 (Act) requires the 
development of recovery strategies for all current 
and future endangered and threatened species. 
Consequently, recovery strategies are required for 
130 endangered and threatened species. Of these, 
one is complete, 116 are in development, and 


Fish and Wildlife Program 


13 have not yet been initiated. The one recovery 
strategy completed was done in August 2008 for 
forest-dwelling woodland caribou. In addition, 
draft recovery strategies for eight of the top 10 
endangered species were posted on Ontario’s 
Environmental Registry for public comment in the 
spring of 2009 and, at the time of our follow-up, 
were targeted for completion later in the year. 


Invasive Species 


Recommendation 4 
To help protect Ontario’s native fish and wildlife 
populations, habitats, and overall biodiversity, the 
Ministry of Natural Resources should: 

@ address knowledge gaps regarding the long- 
term effects of existing invasive species on 
biodiversity; 

e develop action plans that set priorities for the 
prevention, monitoring, and eradication of 
invasive species based on assessments of the 
risks posed by invasive species; 

e evaluate and report on the effectiveness of meas- 
ures taken through these action plans; and 

@ continue to work with the federal government 
to enact more stringent regulations with respect 
to flushing ballast tanks of ocean-going vessels 
before they enter Canadian waters to prevent 
the introduction of destructive invasive species. 


Status 
The Ministry informed us that it was carrying out 
research, which it expected to complete by 2010, 
into the long-term impacts of existing invasive 
species on Ontario’s biodiversity. To help prevent, 
monitor, and eradicate invasive species, the 
Ministry has worked with stakeholders to imple- 
ment public-awareness campaigns, including an 
Invading Species Hotline and a website where the 
public can obtain information about invasive spe- 
cies and report new sightings. In addition, Ontario 
has prohibited the live possession of nine invasive 
fish species, implemented a mandatory training 
program for bait dealers and harvesters, and was 


375 


Chapter 4 « Follow-up Section 4.06 








376 J 2009 Annual Report of the Office of the Auditor General of Ontario 





© | 
eS 
b= 
| — 
(=) 
a 
S | 
® | 
” | 
a} 
= 
= 
= 
rj 
Li | 
C 
+ | 
td 
o 
aa 
a. 
cS 
fe 
oe | 


implementing an improved inland-lake monitoring 
program for aquatic invasive species. In an effort to 
assess the effectiveness of its action plans for inva- 
sive species, the Ministry will carry out an Invading 
Species Angler Awareness survey during 2009. The 
Ministry expects to publish the results of the survey, 
which will examine angler behaviour for washing 
boats, dumping of bait buckets, and the use of live 
bait, in 2010. 

In order to close one path to invasive species, 
the Ministry has continued to work with the federal 
government to enact regulations in 2006 and 2008 
requiring ocean-going vessels to flush their bal- 
last tanks with salt water before entering the St. 
Lawrence Seaway and the Great Lakes. All vessels 
entering the Seaway now are checked through a 
joint Canada/United States inspection program 
to ensure that contaminated ballast water is not 
released in the Great Lakes. 


WILDLIFE MANAGEMENT 
Moose Management 


Recommendation 5 
To assist in maintaining the proper balance between 
keeping moose population levels sustainable and pro- 
viding a reasonable level of hunting opportunities, the 
Ministry of Natural Resources should: 

@ develop and implement a moose management 
policy designed to achieve the overall target 
moose population; 

@ carry out population inventory assessments 
more frequently to more accurately determine 
the current moose population; 

@ ensure that the huntable moose population 
used to determine the number of hunting tags 
issued does not exceed the estimated actual 
population; 

@ more severely restrict hunting in management 
units where the actual number of moose ts sig- 
nificantly below target population levels; and 

@ implement tighter requirements for calf tags in 
all management units with low calf populations. 


Status 
The Ministry informed us that it drafted a new 
policy direction for moose management and has 
undertaken a broad consultation on the moose- 
tag draw system. The Ministry prepared several 
draft documents for public consultation, including 
the Cervid Ecological Framework (cervid species 
include deer, moose, caribou, and elk), the Moose 
Management Policy, the Moose Population Object- 
ive Setting Guidelines, and the Moose Harvest Man- 
agement Guidelines. At the time of our follow-up, 
the public consultation had been completed, and 
the Ministry was reviewing the revised documents 
for final approval. 

During the 2007/08 fiscal year, the Ministry 
increased the frequency of its aerial surveys to 
more accurately determine the moose population. 
These surveys follow the Ministry’s Standards 
and Guidelines for Moose Population Inventory in 
Ontario, which gives the Ministry important data to 
prepare annual moose-harvest plans, assess habitat 
suitability, and evaluate management strategies. 
The aerial survey results contribute significantly to 
the Ministry's determination of allowable moose- 
harvest levels and tag quotas. As such, the Ministry 
has attempted to review all available population 
information to ensure that all management units 
have a sustainable moose population, and that the 
tags issued do not exceed the estimated population 
or target population levels. As noted above, the 
Ministry has drafted new guidelines for population- 
setting and harvest management. These two 
documents are intended to enhance the way that 
moose-population objectives are established and 
to help meet ecologically based moose-population 
goals and objectives to ensure a sustainable moose 
resource. 

The Moose Population Objectives Setting 
Guidelines outline a number of factors that impact 
on moose populations, such as disease, predators, 
and illegal hunting, which ministry staff at each 
management unit will monitor. Staff will also track 
the overall moose population at each unit and take 


immediate action to address any significant popula- 
tion decline. 

The Moose Harvest Management Guidelines 
provide a number of management strategies to be 
employed to restrict the number of moose calves 
harvested in areas with low calf-population dens- 
ities. These strategies include limiting the harvest 
of calves and shortening the calf-hunting season. 


Deer Management 


Recommendation 6 
To assist in maintaining a healthy deer population 
and controlling the spread of disease to more vulner- 
able animals, such as moose, the Ministry of Natural 
Resources should: 

e complete a deer management policy to provide 
strategic direction for managing the increasing 
deer populations; 

@ review its Ontario Deer Harvest Decision Sup- 
port System to ensure that it provides biologists 
with appropriate, complete, and current infor- 
mation to set hunting quotas; and 

e@ work with other jurisdictions to develop better 
detection and monitoring strategies for infec- 
tious diseases. 


Status 
The Ministry informed us that it continuously 
updates its deer-management policies to help inte- 
grate the management of all cervid species (deer, 
moose, caribou and elk) as their ranges continue to 
expand in the province. This has been done through 
the Ministry’s draft Cervid Ecological Framework 
that strategically addresses cervid management 
at the broad landscape and ecological level. It 
consolidates and integrates Ontario’s approach to 
managing cervid species in relation to each other 
with consideration of the broader ecosystems they 
share. The Ministry also informed us that it has 
reviewed its Ontario Deer Harvest Decision Support 
System, a planning tool used to determine appro- 
priate deer-harvest levels to help maintain a healthy 
and balanced deer population. The Ministry will 
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continue to update this system as priorities and 
resources permit. 

The Ministry also indicated that it continues 
to monitor wildlife disease outbreaks in other 
jurisdictions, examines the potential actions it can 
take against emerging diseases not yet detected 
in Ontario, and participates in international and 
inter-agency co-ordination discussions. To enhance 
wildlife-disease monitoring and awareness, the 
Ministry is part of the wildlife disease surveillance 
and response network maintained by the Canadian 
Cooperative Wildlife Health Centre. 


Black Bear Management 


Recommendation 7 
To ensure that black bear populations are maintained 
at sustainable levels in all areas of the province, the 
Ministry of Natural Resources should: 

@ consider sanctions against bear hunters who fail 
to respond to the mandatory provincial mail-in 
surveys, which are needed to obtain accurate 
data to use in setting sustainability guidelines; 
and 

@ take corrective action against tourist outfitters 
who continually exceed the sustainability guide- 
lines for the maximum bear harvest. 


Status 
The Ministry informed us that it has chosen to 
promote and educate bear hunters on the import- 
ance of reporting rather than take sanctions against 
those who fail to report. In this regard, the Ministry 
is educating bear hunters about the importance of 
resource stewardship and the responsibility and 
importance of responding to provincial mail-in 
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surveys. In addition, the Ministry prepared a draft 





Framework for Enhanced Black Bear Management 
in Ontario that provides enhanced public aware- 
ness and understanding of bear management and of 
the importance of hunter reporting for sustainable 
bear-population management. 

At the time of our follow-up, the Ministry 
was developing a new guideline for population 
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objectives and harvest management that will 
include appropriate and consistent measures to 
deal with outfitters who continually exceed the 
harvest of black bears as noted in the sustainability 
guidelines. 


Forest-dwelling Woodland Caribou 


Recommendation 8 
To help protect the threatened forest-dwelling wood- 
land caribou from further deterioration, the Ministry 
of Natural Resources should gather the necessary 
information to finalize and implement its recovery 
strategy on a timely basis. 


Status 
In March 2008, the Ministry established an 
independent Woodland Caribou Science Panel to 
review the scientific basis for its recovery strategy. 
This resulted in the Ministry completing a recovery 
strategy for forest-dwelling woodland caribou in 
August 2008. The recovery strategy forms the scien- 
tific basis for the Caribou Conservation Plan (to be 
released in the near future), which provides broad 
policy direction for the conservation and recovery 
of the forest-dwelling woodland caribou in Ontario 
and outlines the actions the Ministry intends to take 
to protect this species. At the time of our follow-up, 
the Ministry was developing regulations—expected 
to be in place by the end of 2009—to protect the 
woodland caribou habitat under the Endangered 
Species Act, 2007. 


FISHERIES MANAGEMENT 
Commercial Fisheries Management 


Recommendation 9 
To further protect commercial fisheries and fish 
stocks, the Ministry of Natural Resources should: 

@ take appropriate enforcement action when the 
number of fish harvested is above the quotas set 
for sustainability; and 

@ consider developing a bycatch policy to help 
reduce the ecological impact on aquatic 


ecosystems and sustainability of the bycatch 


species. 


Status 
The Ministry informed us that over-quota harvest 
by commercial fishers is being strictly enforced 
through monitoring, inspections, and the laying of 
charges as appropriate. However, with regards to 
aboriginal communities involved in a commercial 
fishery, the Ministry has negotiated consensus 
agreements with First Nation communities that it 
believes will ensure the sustainability of the fish- 
eries resource. 

With respect to the disposition of bycatch 
species—the unintended harvest of certain species 
of fish and other aquatic life—the Ministry has 
conducted a review of commercial fishing bycatch 
strategies in other jurisdictions. Accordingly, the 
Ministry plans to develop a provincial bycatch 
policy by March 2010. 


Recreational Fisheries Management 


Recommendation 10 
To help ensure that recreational fisheries continue to 
be managed in a sustainable manner, the Ministry of 
Natural Resources should develop formal fisheries- 
management plans, along with appropriate time 
frames for implementation. 


Status 
Under the new Ecological Framework for Fisheries 
Management, the Ministry created 20 new Fisheries 
Management Zones (FMZs), which will be man- 
aged to ensure a sustainable recreational fishery. 
Pursuant to the Framework, fisheries regulations 
have been streamlined where appropriate, anda 
scientifically based monitoring program will be 
implemented. To date, the Ministry has developed 
draft fisheries management plans for three FMZs 
and has posted them on Ontario’s Environmental 
Registry for public consultation. According to the 
Ministry, all fishery management plans will be 
in place by the 2013/14 fiscal year. In addition, 
the Ministry has implemented an initial five-year 


fisheries-monitoring program under the new frame- 
work for recreational fisheries management, to be 
completed by the 2012/13 fiscal year. By that time, 
the Ministry plans to have completed assessments 
on more than 1,500 inland lakes to determine fish 
abundance, biodiversity, water quality, and the 
presence of invasive species. 


Fish Stocking Program 


Recommendation 11 
To ensure that the fish-stocking program is effective 
in rehabilitating fish populations and providing 
enhanced recreational angling opportunities, the 
Ministry of Natural Resources should: 

@ perform regular lake surveys and post-stocking 
evaluations to determine whether the stocking 
objectives are being met; and 

@ establish a monitoring program for testing the 
health and quality of fish stocked by its com- 
munity partners. 


Status 
To help determine whether its fish-stocking pro- 
gram is effective and meets stocking objectives, 
the Ministry indicated that it will obtain and 
analyze information through its fisheries monitor- 
ing program for both stocked and non-stocked 
inland lakes. According to the Ministry, there is no 
regulatory authority for a mandatory fish-health 
monitoring program of community hatcheries. 
However, the Ministry participates in a federal— 
provincial initiative led by the Canadian Food 
Inspection Agency to establish a science-based 
regulatory program for disease management called 
the National Aquatic Animal Health Program. This 
program incorporates surveillance and control 
measures for high-risk diseases and will be applied 
to a broad range of sectors, including the Ministry 
and community hatcheries. In addition, the Min- 
istry developed technical bulletins of best practices 
for fish-health management and egg disinfection 
for the use of community hatcheries to reduce the 
spread of disease in stocked fish. 
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ENFORCING COMPLIANCE WITH 
LEGISLATION 


Enforcement Activity 


Recommendation 12 
To help sustain fish and wildlife resources and ensure 
compliance with legislation, the Ministry of Natural 
Resources should determine whether the enforce- 
ment resources allocated are sufficient to achieve the 
enforcement goals established in its risk-based plans. 


Status 
The Ministry implemented a Risk Based Compli- 
ance Framework for the 2008/09 fiscal year to 
guide its enforcement activities. The Framework 
helps to achieve enforcement goals by balancing 
enforcement priorities against fiscal and human 
resources to ensure that enforcement efforts are 
focused on those activities that pose the most 
risk to selected fish and wildlife resources. These 
include the risk to public health and safety, natural 
resources, the economy, and social and cultural val- 
ues. The Ministry indicated that when it identifies 
and sets provincial compliance and enforcement 
priorities, it will review the enforcement resources 
needed to carry out its plans. 


Deployment of Conservation Officers 


Recommendation 13 
To further strengthen its risk-based enforcement 
plan and ensure that fish and wildlife resources are 
adequately protected, the Ministry of Natural Resour- 
ces should review its deployment strategy to determine 
whether conservation officer staffing is sufficient in 
each area to carry out effective deterrent patrols and 
meet local service requirements while recognizing cur- 
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rent funding pressures. 


Status 
As noted in the status of Recommendation 12, the 
Risk Based Compliance Framework is the guiding 
document for enforcement activities in the prov- 
ince. We were informed that, at the time of our 
follow-up, the deployment of conservation officers 
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was being made on the basis of the risk-based plans 
and the Framework, taking into consideration high- 
risk enforcement areas and workload data. The goal 
of the Ministry’s deployment strategy is to deter 
illegal activity and protect resource sustainability. 
During the 2008/09 fiscal year, the Ministry allo- 
cated an additional $2 million to: 
e add five conservation officers to address 
officer-deployment gaps; and 
e fund a multi-year technology project to 
improve mobile enforcement operations. 


Hunting and Fishing Licence Suspensions 


Recommendation 14 
To prevent suspended individuals from obtaining 
hunting and fishing licences or entering the deer and 
moose tag draws while under suspension, the Ministry 
of Natural Resources should improve procedures and 
controls to ensure that its information systems are 
more complete and that suspended hunters are not 
allowed to get moose and deer hunting tags. 


Status 
The Ministry indicated that it made improvements 
to its procedures and information-management- 
system controls to prevent suspended individuals 
from entering deer and moose tag draws or 
obtaining hunting and fishing licences. In addi- 
tion, the Ministry is developing a new automated 
licensing system, to be operational in 2010, that 
will improve its ability to prevent individuals under 
suspension from obtaining a fishing licence or 
hunting tag. 


FISH AND WILDLIFE FUNDING 


Recommendation 15 
Given the decline over the last 20 years in real dollar 
funding for Fish and Wildlife Program activities, the 
Ministry of Natural Resources should formally priori- 
tize its responsibilities for maintaining biodiversity 
and safeguarding Ontario’s fish and wildlife and 
allocate available funding accordingly. 


Status 
Since our audit in 2007, funding for the Fish and 
Wildlife Program has increased by more than 
$21 million to help maintain the province’s bio- 
diversity and to safeguard fish and wildlife resour- 
ces. During the 2008/09 fiscal year, the Ministry 
allocated approximately $25 million in additional 
funding over a number of years for programs such 
as: 
@ species at risk; 
e high-priority fish and wildlife activities; 
@ ecosystem monitoring and assessment for lake 
protection strategies; 
e stewardship programs; 
e@ partnership funding for the Great Lakes basin 
ecosystems; 
® monitoring programs for endangered species; 
and 
e the enforcement program. 


MEASURING AND REPORTING ON 
EFFECTIVENESS 


Recommendation 16 
The Ministry of Natural Resources should develop 
more comprehensive indicators for measuring and 
reporting on the Fish and Wildlife Program’s effect- 
iveness in ensuring that Ontario’s fish and wildlife 
resources are healthy, diverse, and sustainable for the 
use and enjoyment of the people of Ontario. 


Status 
According to the Ministry, some progress has been 
made to develop more comprehensive indicators for 
the effectiveness of the Fish and Wildlife Program. 
Through its involvement with the Ontario Bio- 
diversity Council’s effort to prepare Ontario’s first 
biodiversity report in 2010, the Ministry is also to 
report on the success of its programs to ensure that 
fish and wildlife resources are healthy and sustain- 
able. At the time of our follow-up, the Ministry was 
also in the process of establishing outcome-based 
performance measures for Great Lakes sustainabil- 
ity, to be in place for the 2010/11 fiscal year. 
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Background 


Established in 1967, GO Transit’s commuter net- 
work is a vital part of the GTA transportation sys- 





tem, linking Toronto with the surrounding regions 
and serving a population of more than 5 million. 
On a typical weekday, GO Transit trains carry about 
172,000 passengers (165,000 in 2007) and its buses 
carry an additional 28,500 passengers (30,000 in 
2007). As of March 31, 2009, GO Transit had over 
1,700 full-time-equivalent employees (1,200 as of 
March 31, 2007). Its annual operating expendi- 
tures, including amortization on capital assets, 
were approximately $507 million in 2008/09 
($375 million in 2007), of which about $284 mil- 
lion ($250 million in 2007) is recovered from pas- 
senger fares, with the province subsidizing the rest. 
In our 2007 Annual Report, we noted that the 
demand for GO Transit services was growing 
rapidly, with more than a 65% increase in rail pas- 
sengers over the previous 10 years. Until recent 
years, GO Transit’s on-time performance was in 
the mid-90% range, but delays and overcrowding 
had become increasingly common. During our 
audit, between October 2006 and February 2007, 
GO Transit’s on-time performance was only about 
85%. While GO Transit had taken some action to 
address this, more needed to be done to meet ser- 
vice demand and provide reliable rail services. 
Specifically, GO Transit’s capital expenditure 
plan was based not on projected ridership growth 
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but mainly on expected government funding levels. 
Without a more comprehensive analysis of future 
demand, there might not be sufficient infrastruc- 
ture to accommodate future growth in passenger 
volumes. Some areas of the GTA could continue to 
experience serious capacity issues and persistent 
problems with customer service. 

At the time of our audit, 70% of the track that 
GO Transit operates on was privately owned. GO 
Transit had limited means to deal with what it con- 
sidered to be high rates, restrictive covenant provi- 
sions, and controls over service levels that were 
imposed by the private railways. As well, over the 
next ten years, approximately $475 million was to 
be spent by GO Transit on improvements in rail ser- 
vice on the privately owned rail corridors. Although 
the railways were to maintain ownership of and 
control over the improved infrastructure, there 
was little guarantee that GO Transit would receive 
improved service in return. Because the regula- 
tion of railways falls under federal legislation, 

GO Transit needed to work more closely with the 
provincial Ministry of Transportation to ensure that 
representations made to the federal government 
better safeguard taxpayer-funded railway projects 
and to ensure adequate access to commuter railway 
service for the public. 

With respect to GO Transit’s proof-of-payment 
fare system, 99% of all riders inspected paid the 
appropriate fare. However, approximately 60% of 
all fare inspections were done on non-rush-hour 
trains with significantly fewer passengers. 
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With respect to the acquisition of goods and 
services, we found: . 

e Although the scope of work for two significant 
program-management consultant contracts 
clearly extended over several years, in 
selecting the successful consultant, GO Transit 
requested proposals for work spanning only 
12 and 17 months. In the first case, a contract 
was awarded to a consortium for $247,000, 
repeatedly extended for another seven years, 
and totalled more than $25 million as of the 
time of our audit. Similarly, in the other case, 
a contract initially awarded for $2.3 million 
was subsequently extended for three years at 
an additional cost, as of the time of our audit, 
of $15.2 million. 

e Including the extensions to the contract 
referred to above, more than 60 major amend- 
ments were made to contracts, totalling 
almost $70 million, or an increase of about 
75% of the original contracts’ values, in the 
three years from 2004 through 2006. 

e@ There were numerous instances of suppliers 
being selected without a competitive process, 
including over $8.6 million for 170 single- 
sourced consultant contracts over two years. 

We made a number of recommendations for 

improvement and received commitments from 
GO Transit that it would take action to address our 
concerns. 


Status of Recommendations 





In March 2009, the Ontario government introduced 
the Greater Toronto and Hamilton Area Transit 
Implementation Act, 2009, to merge GO Transit 

and Metrolinx into a single regional transit agency. 
Formerly known as the Greater Toronto Transporta- 
tion Authority, Metrolinx is a Crown-controlled 
agency created by the Ontario government in 2006 
to develop and implement an integrated transporta- 
tion plan for the Greater Toronto and Hamilton 


area. Under the legislation, the government is 
responsible for appointing a board of directors 
for the merged entity, known as Metrolinx, which 
is to plan, build, and operate transit across the 
Greater Toronto and Hamilton area. The merger 
became official in May 2009. For the purpose of this 
follow-up, we are referring to the actions taken by 
GO Transit prior to the merger. We recognize that 
direction for future implementation of our recom- 
mendations will need to come from Metrolinx’s new 
Board and management. 

According to information received from 
GO Transit in spring 2009, significant progress has 
been made in addressing many of our recommen- 
dations. But the implementation of others, such as 
those involving infrastructure improvements and 
outstanding fines collection, will require more time. 
The status of action taken on each of our recom- 
mendations is described in the following sections. 


SERVICE DEMAND AND ON-TIME 
PERFORMANCE 


Reasons for Train Delays 


Recommendation 1 
In order that appropriate and timely action is taken 
to ensure the on-time performance of trains, GO Tran- 
sit should: 

e formalize the practice of periodically conducting 
individual trip reviews; 

@ follow through with its commitment to carry 
out a review of systemic issues leading to train 
delays and develop and implement an action 
plan with timelines to address each significant 
systemic issue; and 

@ conduct an updated customer satisfaction sur- 
vey to obtain input from customers on ways to 
improve service. 


Status 
GO Transit informed us that the on-time perform- 
ance of all train trips had been formally reviewed 
on an individual basis. Its Board of Directors 
received regular reports on on-time performance, 


and it was posting on its public website its monthly 
on-time performance for the previous twelve 
months for individual train trips. This would allow 
customers to compare the past reliability of particu- 
lar trips when making their travel plans. 

In addition, GO Transit had identified systemic 
delay issues and acted upon a number of major 
factors contributing to them. This had resulted in 
changes to operations and procedures. However, 
GO Transit also indicated that many of the miti- 
gation measures for systemic problems involve 
infrastructure improvements that would take a 
number of years to make. These measures were 
incorporated into GO Transit’s multi-year capital 
plans and the Board was being updated quarterly 
on the status of many of these major infrastructure 
programs. 

With respect to obtaining input from customers 
on ways to improve service, GO Transit informed 
us that it has been continuously monitoring and 
evaluating customer satisfaction. For example, it 
had established a Customer Service Advisory Com- 
mittee composed of GO Transit customers. Monthly 
reports on customer complaints and feedback were 
presented to the Board. 

GO Transit had conducted a number of surveys 
to track customer characteristics, trends, and satis- 
faction. One of the surveys conducted in 2008 was 
to ensure that GO Transit management was aware 
of the most significant factors affecting riders’ 
decisions whether to take GO Transit for their com- 
mute. The results indicated that passenger growth 
and loyalty would continue but keeping trains and 
buses running on time is of paramount importance. 

GO Transit informed us that customer satisfac- 
tion surveys, which used to be done every two to 
four years, will now be undertaken annually. 


Capital Planning to Address Growth 


Recommendation 2 
To ensure that an effective strategy is in place to 
address growing passenger demand, GO Transit 
should establish a more comprehensive capital plan- 
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ning process that takes into consideration such factors 
as passenger growth by individual corridor and the 
impact of different funding levels on meeting service 
demand. 


Status 
According to GO Transit, this recommendation 
would be addressed through the Regional Trans- 
portation Plan (RTP) prepared by Metrolinx, which 
was released in late September 2008. The Metrolinx 
RTP establishes what infrastructure is required to 
meet transportation demand in the GTA and Ham- 
ilton area, and thereby provides the foundation for 
a comprehensive capital planning and funding pro- 
cess. GO2020—GO Transit’s strategic plan in place 
at the time of our follow-up—was consistent with 
Metrolinx’s RTP and was being used as the basis for 
GO Transit’s current capital plan in the interim. 

In addition, GO Transit had already embarked 
on a billion-dollar infrastructure-expansion pro- 
gram to improve its rail system to help alleviate 
the congestion affecting many GO Transit train 
stations. 


Track Congestion and Right of Access 


Recommendation 3 
To ensure that the interest of the public is adequately 
protected, GO Transit should work proactively with 
the province to ensure the public’s right of access to 
economical and efficient railway service. 


Status 
GO Transit informed us that this work was in 
progress at the time of our follow-up. Working with 
the province, GO Transit had met with CN to start 
negotiations to secure GO Transit ownership of 
various rail corridors. GO Transit acquired two cor- 
ridors in the 2008/09 fiscal year. The province has 
established a capital fund to accommodate further 
acquisitions and this strategy was reinforced in the 
GO2020 plan. 
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Agreements with Host Railways and 
Suppliers 


Recommendation 4 
To ensure reliable train service, GO Transit should: 

e work more effectively with service providers to 
address persistent delays attributed to them, 
monitor progress toward reducing the delays, 
and take appropriate action; and 

@ review the terms of the agreements with service 
providers and, where possible, negotiate appro- 
priate changes to future agreements to enhance 
performance and accountability. 


Status 
GO Transit informed us that it had held regular 
meetings with the host railways and major suppli- 
ers to identify on-time performance issues, contrib- 
uting factors, and potential solutions. This process 
had yielded a number of procedural improvements 
and operational adjustments, as well as the initia- 
tion of capital projects to help improve on-time 
performance. In addition, new contracts were being 
structured to focus on the functional requirements 
needed to improve performance and customer 
service. 

GO Transit indicated that it had awarded 
competitively bid contracts for train operation and 
train maintenance that are performance-based 
and include provisions for damages. The major 
contracts with host railways are multi-year agree- 
ments. As they come due for renegotiation, GO 
Transit intends to seek terms that enhance perform- 
ance and accountability. GO Transit’s strategy 
of acquiring corridors was also a response to the 
limited leverage it had in negotiating with the host 
railways. 


ACQUISITION OF GOODS AND SERVICES 


Recommendation 5 
To ensure that value for money is received and 
GO Transit’s acquisition processes are regarded as 
fair, open, and transparent, GO Transit should: 


@ follow its internal policies, which require a com- 
petitive selection process in acquiring goods and 
services; 

@ monitor contracts for adherence to the original 
price and consider obtaining a separate tender 
for any significant change in the scope of work 
in the original contract; 

e ensure that contracts have firm ceiling prices, 
whenever possible; 

@ conduct a long-term needs analysis on the costs 
and benefits of hiring consultants and consider 
alternatives, such as hiring and training staff 
instead of using consultants; and 

@ strengthen the terms of contracts with suppliers 
to ensure satisfactory and timely performance 
and take appropriate action to ensure that sup- 
pliers adhere to contract terms. 


Status 
GO Transit informed us it has strengthened its 
internal policies. It has modified its procurement 
policies to enforce the competitive bid process 
and highlight that contract amendments and sole- 
sourced negotiated contracts are the two “least 
favourable” methods of contracting for new work. 
Each contract change that is brought forward for 
approval is challenged and subject to lengthy dis- 
cussion by senior staff. In addition, GO Transit had 
instituted on all contracts a firm “ceiling” or “stipu- 
lated” price for the work specified in the contract. 
According to GO Transit, it had established an 
ongoing process to review its consulting needs. 
The existing large assignments contracted out to 
program management consultants were scheduled 
to end on March 31, 2010. Reorganization within 
GO Transit was in progress, which would allow 
for a smooth transition of responsibilities from the 
program management consultants to GO Transit 
staff. GO Transit told us that bringing this work 
in-house was made possible by the Metrolinx RTP 
and the province’s Move-Ontario 2020 Program. 
Because the program committed more stable fund- 
ing, GO Transit indicated that it could hire staff for 
work that was often previously contracted out due 


to uncertainty as to whether the necessary funding 
would be available. 

With respect to strengthening the terms of 
contracts with suppliers to ensure satisfactory and 
timely performance, GO Transit’s legal depart- 
ment had reviewed the terms and conditions of all 
contract documents, including construction-related 
and consulting agreements. It was in the process of 
enhancing existing standardized forms and imple- 
menting additional standardized agreements. 


ENSURING PAYMENT OF FARES 


Recommendation 6 
To ensure that inspection and collection efforts are 
effective and consistent in enforcing payment of fares, 
GO Transit should: 

@ review and make appropriate revisions to its 
inspection guidelines relating to when a fine 
should be levied on passengers who evade pay- 
ing their fares; 

e@ make sure inspection coverage and enforce- 
ment actions comply with internal inspection 
standards; 

@ monitor the results of inspections and take cor- 
rective action, where necessary; 

e develop a policy with respect to repeat offenders; 
and 

e work with the Ministry of the Attorney General 
and municipalities to establish a more effective 
and accountable system for collecting fines. 


Status 
GO Transit informed us that it developed guidelines 
for staff outlining when fines should be levied on 
fare-evading passengers. These guidelines were 
included in a refresher training program delivered 
to all enforcement staff in May 2008. The super- 
visory group responsible for monitoring compliance 
also received additional training. 

Internal inspection standards for 2009/10 were 
being finalized at the time of our follow-up. In mid- 
2008, GO Transit hired a staff dedicated to schedul- 
ing and tracking inspection resources to ensure 


co Transit (EE 


that internal inspection standards are maintained. 
Performance results were being tracked ona 
monthly basis to determine what corrective actions, 
if any, should be taken. The policy guidelines for 
fare enforcement had been updated to include a 
procedure to deal with repeat offenders. 

In addition, GO Transit informed us that it had 
begun developing a model of administrative fees to 
deal with fare evaders and persons in contravention 
of parking regulations. This model requires legisla- 
tive amendment but has been identified as a key 
component in improving the fine-payment process. 
It would see the process moved out of the court 
system and become a matter preferably resolved 
directly by GO Transit and the customer. This pro- 
cess is expected to improve the effectiveness of fare 
collection and reduce the number of outstanding 
fines. 

With respect to the collection of fines, GO Tran- 
sit’s legal services met with all relevant court 
managers in February 2008. They determined 
that legislative amendment is needed to increase 
the effectiveness of collecting outstanding fines. 
GO Transit staff has approached the Ministry of 
Transportation to address this matter. 


SAFETY AND SECURITY 


Recommendation 7 
To further enhance the safety and security of passen- 
gers, GO Transit should: 

@ perform periodic systemic analysis of past safety 
and security incidents to determine whether 
measures can be taken on certain types of com- 
monly recurring risks; 

@ formally analyze and report on the effectiveness 
of its simulated security exercises; and 

e implement safety and security measures identi- 
fied through audits on a timely basis. 


Status 
GO Transit informed us that, at the time of our 
follow-up, it had started gathering statistics on 
security-based activities and was moving forward 
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on the acquisition of an Incident Management 
System. 

In addition, GO Transit had completed a pre- 
liminary review of existing procedures related to 
parking lot security and had implemented various 
changes, including increasing the visibility and fre- 
quency of random station patrols, sharing vehicle 
crime statistics with local police services, and 
reviewing the use of closed-circuit TV recording. 

GO Transit also completed a simulated tabletop 
security exercise in March 2008 and subsequently 
analyzed and reported on the effectiveness of the 
exercise. Staff presented a report, which included 
recommendations, to senior management. 

According to GO Transit, any follow-up actions 
recommended in audits and/or reports are tracked 
to completion. 


BOARD GOVERNANCE 


Recommendation 8 
To provide more effective governance, GO Transit’s 
Board of Directors should: 

® approve a formal strategic plan setting GO 
Transit’s strategic direction and share it with 
the Ministry of Transportation and the Greater 
Toronto Transportation Authority (GTTA); 

e establish a committee structure that supports 
the Board with more detailed review of and 
advice on significant matters relating to overall 
governance and oversight; 

@ ensure more effective oversight of GO Transit’s 
overall performance, as well as of specific 
operational issues, such as procurement and 
contract management; and 

@ consider adopting certain governance best prac- 
tices such as enhanced performance-evaluation 
processes and a more formal orientation for new 
Board members, as well as periodic governance- 
training workshops. 


Status 
According to GO Transit, governance practices, 
including the establishment of various committees 


to ensure effectiveness and efficiency, have been 
put in place and continued at the new Metrolinx 
Board. 

The now-dissolved GO Transit Board informed 
us that it had undertaken a more formal approach 
in developing a strategic plan (GO2020). A number 
of comprehensive meetings took place with the 
Board directing and guiding staff to create the cor- 
poration’s next plan. 

The former GO Transit Board had established 
a Risk Management Committee and a Governance 
and Compensation Committee. These committees 
provided regular in-depth analysis of inherent risks 
to the organization, ensured that all new directors 
receive comprehensive orientation and training, 
assessed the effectiveness of the Board of Directors 
and its committees, and made recommendations 
with respect to compensation issues. 


AGENCY ACCOUNTABILITY 


Recommendation 9 
To fulfill its accountability requirements to the Min- 
istry of Transportation, GO Transit and the Ministry 
of Transportation should work together to finalize 
a Memorandum of Understanding and produce an 
annual business plan and annual report in compli- 
ance with provincial policies and guidelines. 


Status 
GO Transit informed us that a Memorandum of 
Understanding had been approved by the former 
GO Transit Board and the Ministry of Transporta- 
tion. Effective for the 2010/11 fiscal year, a business 
plan and annual report are to be produced in com- 
pliance with provincial policies and guidelines. At 
the time of our follow-up, anew Memorandum of 
Understanding between Metrolinx and the Ministry 
of Transportation was being drafted as a result of 
the merger. 







Section 


Background 


Hazardous wastes include a broad range of sub- 
stances such as waste acid, contaminated sludge, 
photo-finishing and other chemicals, motor oil, and 
discarded batteries. The Ministry of the Environ- 
ment (Ministry) is responsible for ensuring that 
hazardous waste is collected, stored, transported, 
treated, and disposed of with due regard for 
the environment and public health. Excluding 
household hazardous waste, Ontario produces 
approximately 400,000 tonnes of hazardous waste 
annually. In the 2008/09 fiscal year, the Ministry 
spent $16.1 million ($14.6 million in 2006/07) on 
its Hazardous Waste Management Program. 

During our 2007 audit, we found that, partly 
owing to continuing problems with a computer 
system implemented in 2002, the Ministry did 
not yet have adequate monitoring and inspection 
procedures in place to ensure compliance with 
legislation and regulations aimed at protecting the 
environment from the risks posed by hazardous 
wastes. The system’s weaknesses limited the ability 
of staff to monitor effectively the volume of hazard- 
ous waste activity in the province and contributed 
to many of the following concerns: 

e We identified more than 5,000 organizations 

that registered as hazardous waste generators 
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in 2004 but not in 2005. Yet the Ministry had 
not determined whether these organizations 
were still generating hazardous waste and 
properly disposing of it. 
Certificate-of-approval applications from 
hazardous waste carriers and receivers that 
want to establish or expand a facility are 
reviewed by the Ministry to ensure that appli- 
cant operations will not adversely affect the 
environment. As of January 2007, we found 
that 50% of the applications remaining to be 
processed had been in the assessment stage 
for more than a year, and 20% for more than 
three years. 

In 2005, there were more than 26,000 ship- 
ments of hazardous waste where the quantity 
received was less than the quantity shipped by 
the generator. The difference was greater than 
10% in half these shipments, with no explana- 
tion or follow-up regarding the discrepancy. 
Consequently, there was a risk that significant 
amounts of hazardous waste were not being 
disposed of properly. 


e We identified almost 900 registered hazard- 


ous waste generators that apparently had not 
shipped any hazardous waste for the last three 
consecutive years as evidenced by the absence 
of manifests, which must accompany all 
shipments of hazardous wastes. The absence 
of manifests could indicate that hazardous 
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wastes were being shipped and disposed 
of inappropriately if they were not being 
accumulated on-site. 

e As of April 2007, the Ministry held $150 mil- 
lion in financial assurance from over 700 
carriers and receivers of hazardous wastes. 
Financial assurance is required to ensure that 
the government does not end up paying for 
hazardous waste cleanup. However, the finan- 
cial assurance collected is often inadequate. 
For example, one chemical company provided 
financial assurance totalling $3.4 million for a 
landfill site, but the site experienced problems 
with leakage, and cleanup costs have been 
estimated at $64 million. 

e Although the Ministry performed a significant 
number of inspections of hazardous waste gen- 
erators, carriers, and receivers, its selection of 
facilities for inspection was often not based on 
risks posed to the environment. In the last five 
years, at least, the Ministry had performed no 
inspections at 11 of the 30 largest hazardous 
waste-generating facilities in the province. In 
addition, ministry inspectors found a signifi- 
cant level of repeat non-compliance, but had 
given the repeat violators more severe penal- 
ties only 20% of the time. 

We made a number of recommendations for 
improvement and received commitments from the 
Ministry that it would take action to address our 
concerns. 


Status of Recommendations 





According to information received from the Min- 
istry, action has been taken on all of the recom- 
mendations made in our 2007 Annual Report, with 
substantial progress being made on many of them. 
The status of actions taken on each of our recom- 
mendations is as follows. 


HAZARDOUS WASTE MANAGEMENT 
OPERATIONS 


Registration of Hazardous Waste 
Generators 


Recommendation 1 
To ensure that all hazardous-waste-generating facili- 
ties are registered as required, the Ministry of the 
Environment should: 
® consider implementing deterrents to encourage 
generators to register by the legislated deadline 
and help reduce the significant volume of non- 
compliance; and 
@ inform district offices of all generators that do 
not register by the legislated deadline and fol- 
low up to ensure that they either register or no 
longer generate hazardous waste. 


Status 
The Ministry informed us that it had introduced a 
series of procedures to ensure that all unregistered 
generators receive adequate deadline notifications 
and, if necessary, timely follow-up: three months 
prior to the registration deadline, the Ministry 
sends notices to generators reminding them of the 
legislated registration deadline. Subsequent to the 
deadline, the Ministry sends notices to unregistered 
generators.notifying them that their registration 
has expired and that, if they generate any hazard- 
ous waste that requires shipping or disposal, they 
will be out of compliance. 

The Ministry informed us that the current 
Hazardous Waste Information Network (HWIN) 
system generates exception reports, which list all 
shipments originating from unregistered generators 
or handlers not authorized by their certificates to 
handle hazardous waste. In March 2008, the Min- 
istry implemented a new process where a dedicated 
staff member follows up on all exception reports. A 
standard operating procedure was created to assist 
ministry staff in this follow-up process. It outlines 
the steps to take to identify and follow up on 
cases of non-compliance of a generator, carrier, or 
receiver. If compliance cannot be achieved through 


this process, the exception report is provided to 

the district office so it can conduct a site visit. The 
exception reports resulting from generators account 
for approximately 94% of all exception reports of 
generators, carriers, and receivers. The Ministry 
indicated that, by the end of April 2009, more than 
80% of these exception reports had been followed 
up on and resolved. 

As an additional compliance measure for 
unregistered generators that ship waste, in 
April 2008, the Ministry sent out a letter to all 
carriers of hazardous waste that were actively 
transporting waste as documented in HWIN in 
2007. The letter reminded the carriers that picking 
up hazardous waste from unregistered generators 
is a violation of the regulation and their certificate 
of approval. In April 2009, a similar letter was sent 
to all active carriers of hazardous waste that were 
actively transporting waste as documented in HWIN 
in 2008. 

The Ministry indicated that there were discus- 
sions on imposing financial deterrents and penalties 
to encourage generators to register by the legislated 
deadline in order to help reduce the volume of non- 
compliance. At the time of our follow-up, a final 
decision had not been made. 

The Ministry also indicated that, in May 2009, 
it began sending “potential violation letters” to 
expired generators giving them 30 days to register 
their site or close their accounts, after which time 
compliance action would be taken, which may 
include referral to the Investigations and Enforce- 
ment Branch of the Ministry. When these letters 
were mailed out, district offices were notified of the 
unregistered companies located in their geograph- 
ical area so they could follow up on them. 


Certification of Hazardous Waste Carriers 
and Receivers 


Recommendation 2 
To help ensure that certificates of approval are in 
place for all carriers and receivers of hazardous waste 
and that certificate applications are properly assessed 
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and approved on a timely basis, the Ministry of the 
Environment should: 

e@ implement procedures to ensure that all carriers 
and receivers of hazardous waste are holders of 
the legally required certificates of approval; 

e ensure that all required documentation has 
been submitted and is on file before issuing a 
certificate; 

@ consider options for the submission of inde- 
pendent third-party evidence that applica- 
tion proposals comply with legislation and 
adequately protect the environment, as is done 
for other environmentally sensitive programs 
such as mines and forestry; 

e enhance the functionality of the Integrated Div- 
istonal System to interface with other program 
systems and to distinguish hazardous waste 
certificates from other program certificates; and 

e include all existing certificates and reporting 
requirements in its management information 
system. 


Status . 
In March 2008, the Ministry implemented a new 
process where a dedicated staff member follows up 
on all exception reports on carriers and receivers. 
These exception reports are generated by the HWIN 
system and identify all carriers and receivers with- 
out a valid certificate of approval. If this follow-up 
does not result in compliance, the exception report 
is provided to the district office so it can conduct a 
site visit and/or an inspection. 

The Ministry informed us that, in addition, haz- 
ardous waste carriers and receivers had been added 
to the annual inspection plan to ensure that they 
are operating within the conditions of their certifi- 
cate of approval. Subsequent to our 2007 audit, the 
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Sector Compliance Branch and the Investigations 
and Enforcement Branch of the Ministry conducted 
sampling on carriers to verify the contents of 
vehicles that transport hazardous waste, and the 
Sector Compliance Branch now includes vehicle 
inspections in its annual work plans. 
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The Ministry informed us that it had rearranged 
the Application Processing Unit into specific groups 
by types of applications (air, water, wastewater, and 
waste) to better screen out incomplete applications. 

The Ministry also updated the certificate-of- 
approval application guidance document and the 
application form. The updated guidance document 
clarifies what documents are required when an 
application is submitted. The updated application 
form is an interactive e-form that provides a list of 
supporting documentation required based on the 
type of application that is being submitted. If an 
applicant does not complete all mandatory fields, 
the application form will indicate that the applica- 
tion is incomplete. In March and April 2009, the 
Ministry also published sample application pack- 
ages to illustrate the expected format and content 
of the application forms. 

The Ministry informed us that, at the time of our 
follow-up, third-party submissions had become a 
mandatory component in the application process 
for a comprehensive certificate of approval for non- 
hazardous waste disposal. The Ministry was consid- 
ering extending this type of certificate to hazardous 
waste sites. The Ministry also stated that it would 
continue to look for ways to capitalize on the use of 
submissions of independent third-party evidence 
that indicates application proposals comply with 
legislation and adequately protect the environment. 

The Ministry informed us that upgrades had 
been made to the Integrated Divisional System to 
enable more comprehensive description and char- 
acterization of waste disposal sites by type (landfill 
or processing site) and by the type of wastes (haz- 
ardous or municipal waste) that are managed. This 
enhancement allows better tracking, reporting, 
and responding to waste disposal site applications. 
In November 2007, the Ministry upgraded the 
Integrated Divisional System to send out automatic 
email notifications to the local district office when- 
ever the Approval Branch decides to issue, return, 
or revoke an application for a hazardous waste 
certificate of approval. 


We were informed that, in 2007, the Ministry 
reviewed the HWIN database and identified those 
sites that had received hazardous waste. All sites 
identified through HWIN were cross-referenced to 
the Integrated Divisional System and were added 
to the Integrated Divisional System if the site was 
not previously entered. The Ministry also indicated 
that improvements have been made to increase the 
ability of the Integrated Divisional System to track 
whether a company has provided the appropriate 
amount of financial assurance and identify when a 
financial-assurance reassessment is due. 


Monitoring Hazardous Waste Shipments 


Recommendation 3 
To ensure that hazardous waste shipments are prop- 
erly monitored to minimize risk to the public and the 
environment, the Ministry of the Environment should: 

@ follow up on all significant waste shipments that 
originate with unregistered generators; 

e investigate all hazardous waste carriers and 
receivers that are not authorized by their certifi- 
cates of approval to handle the hazardous waste 
manifested; 

@ review all registered generators with no manifest 
activity for extended periods of time to ensure 
that they are not involved in unauthorized 
waste shipment and disposal; 

@ investigate significant discrepancies between 
the amount of hazardous waste shipped and the 
amount received; and 

@ implement procedures to ensure that hazardous 
waste temporarily stored at a receiving facility 
is not double-counted in determining the total 
hazardous waste produced in Ontario each year. 


Status 
In March 2008, the Ministry implemented a new 
process where a dedicated staff member follows 
up on all generator, carrier, and receiver excep- 
tion reports. As of April 23, 2009, a total of 4,747 
exception reports for generators, carriers, and 
receivers had been generated, and more than 3,800 


of them had been closed. The centralization of the 
exception report follow-up process has allowed for 
consistent and timely resolution, and resulted in a 
34% decrease in exception reports in comparison to 
the previous fiscal year. 

The Ministry indicated that, in order to investi- 
gate all hazardous waste carriers and receivers that 
are not authorized by their certificates of approval 
to handle the hazardous waste manifested, based 
on the centralized exception report, inspections of 
these carriers and receivers took place where neces- 
sary to verify if the manifested waste corresponds 
to that on the certificate of approval. 

Among the receiver exception reports, the 
majority of them are due to incorrect waste class 
information provided on the paper manifest by the 
generator. The Ministry has instructed receivers to 
ensure that all manifested waste has been properly 
documented on the manifest. 

Carriers that are found to be transporting haz- 
ardous waste not approved within their certificate 
of approval are contacted and instructed to apply 
for a certificate-of-approval amendment if they 
intend to continue to transport the hazardous 
waste in question. If companies continue to operate 
outside of their certificate of approval, a provincial 
officer can issue an order to them to cease oper- 
ations until an amended certificate of approval is 
issued or an agreement to cease transportation of 
the hazardous waste in question has been reached. 
If companies continue to transport the material in 
contravention of an order, penalties will be levied. 

To ensure that registered generators with no 
manifest activity for an extended period of time are 
not involved with unauthorized waste shipment 
and disposal, the Ministry has reviewed and fol- 
lowed up on companies with no manifest activity to 
ensure that waste is being disposed of properly. 

The Ministry also indicated that it has followed 
up with companies where manifest discrepancies 
were occurring and found that discrepancies occur 
because generators simply estimate the amount 
of waste they ship, and the waste is only weighed 
at the receiving site. The manifest document does 


Hazardous Waste Management . 391 


not require actual weight, and the current regula- 
tion only requires the receiver to track the amount 
of waste on a daily or weekly basis. The Ministry 
noted that approximately 60% of these weight dis- 
crepancies were found in the medical industry. To 
follow up on the discrepancies, from October 2007 
to January 2008, the Ministry has completed 46 
inspections of hospitals in Ontario for compliance 
in handling biomedical waste. There was no evi- 
dence of waste being lost in transit. 

To prevent double-counting of hazardous waste 
temporarily stored at a receiving facility, the Min- 
istry has developed a procedure to produce reports 
on hazardous waste shipped that excludes waste 
shipped from receiving facilities with temporarily 
stored waste. 


Storage and Disposal of Hazardous Waste 


Recommendation 4 
To help reduce the substantial risk that the disposal 
and storage of hazardous waste pose to the public and 
to the environment, the Ministry of the Environment 
should develop a strategy to resolve the concerns that 
have delayed regulatory amendments designed to 
reduce the risks posed by medical waste and PCBs. 


Status 

The Ministry informed us at the time of our 
follow-up that the guideline on the management of 
biomedical waste in Ontario was being updated. In 
October 2008, the Ministry posted a revised version 
of the guideline to the Environmental Registry for 
a 60-day public comment period, which closed on 
December 22, 2008. At the time of our follow-up, 
stakeholder comments were being analyzed and 
additional stakeholder consultations were taking 
place. The Ministry indicated that the revised 
guideline would be completed and posted to the 
Environmental Registry. 

The Ministry told us that it was actively address- 
ing the removal of PCBs from the Pottersburg 
PCB storage site, the largest PCB storage site in 
Ontario. In February 2009, the Ministry engaged 
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a contractor to decommission and restore the Pot- 
tersburg site. The decommissioning and restoration 
of the site was expected to be completed by the end 
of 2010. 


Household Hazardous Waste 


Recommendation 5 
To build on its recent initiatives for the disposal of 
household hazardous wastes, the Ministry of the 
Environment should work with Waste Diversion 
Ontario and municipalities on a province-wide strat- 
egy for reducing the impact of household hazardous 
waste on the environment. 


Status 
In December 2006, the Ministry directed Waste 
Diversion Ontario (WDO) to develop an industry- 
funded diversion program plan, called the Munici- 
pal Hazardous or Special Waste (MHSW) Program, 
to enhance the proper management of MHSW. The 
program addresses both the collection of unused 
material as well as consumer education regarding 
proper handling and use of it. The program was 
received and posted on the Environmental Registry 
for a 30-day public comment period on June 11, 
2007. At the time of our follow-up, the Ministry 
indicated that Phase I of the MHSW Program was 
approved by the Minister in February 2008. The 
industry has developed and managed the program 
since July 2008 and it was officially launched in 
January 2009. The goal of the program is to divert 
common hazardous consumer products, such as 
paints and solvents, from being disposed of in land- 
fills or sewers. 

We were informed that, at the time of our 
follow-up, WDO and Stewardship Ontario were 
developing program plans for phases II and III of 
the program, and would submit a consolidated pro- 
gram plan for all three phases to the Ministry by the 
end of July 2009. Phase II is to include additional 
MHSW materials not included in Phase I. The final 
phase, Phase III, is to include the remaining MHSW 
materials that meet the municipal hazardous waste 
definition. The Minister’s approval for the consoli- 


dated MHSW program plan will be required before 
it can commence. 


INFORMATION AND REPORTING 
SYSTEMS 


Hazardous Waste Information System 


Recommendation 6 
To ensure that management and inspection staff have 
reliable and relevant information for monitoring 
whether hazardous waste is transported and disposed 
of in accordance with legislation, the Ministry of the 
Environment should: 
e identify its key information needs; 
e consider how other jurisdictions obtain similar 
information; and 
@ formulate a business case that outlines the cost 
and benefits of various alternatives to meeting 
its information requirements. 


Status 
To ensure that management and inspection staff 
have reliable and relevant information to monitor 
the transportation and disposition of hazardous 
waste, the Ministry had identified its informa- 
tion needs and, at the time of our follow-up, was 
developing a multi-year project plan for a new busi- 
ness architecture and system solution to support 
the hazardous waste program. The project plan is to 
explore alternative means to improve information 
management. 

The Ministry indicated that it had collected 
information on how other leading jurisdictions 
collect and manage hazardous waste information. 
The research on other jurisdictions revealed that 
Ontario is a leader in many aspects of its hazardous 
waste program. Ontario is the only jurisdiction in 
Canada to require annual registration of hazardous 
waste generators. Ontario also has the most com- 
prehensive manifesting and verification system and 
has the first fully electronic information system. 
The Ministry was intending to update its jurisdic- 
tional analysis to include additional jurisdictions by 
mid-October 2009. 


The Ministry reported that it had evaluated 
the options of enhancing the current system or 
developing a new system. Upon performing the 
cost/benefit analysis, the Ministry concluded that 
the functionality and reliability of a new system 
would address the public interest and the Ministry’s 
long-term needs. A new system would also be sus- 
tained with lower maintenance costs. A new system 
is to be developed on the current platform with 
improved reliability, functionality, and flexibility to 
adapt to regulatory changes. 


Measuring and Reporting Program 
Effectiveness 


Recommendation 7 
To enhance ministry decision-making and provide the 
public with information on its success in managing 
hazardous waste, the Ministry of the Environment 
should: 

e establish more comprehensive performance 
measures for hazardous waste management; 

@ review the performance measures for hazardous 
waste management used by other jurisdictions 
for applicability to Ontario; and 

e publicly report on those measures. 


Status 
The Ministry informed us at the time of our follow- 
up that it had identified three performance meas- 
ures that it was planning to report on publicly in fall 
2009. On the basis of the current database, the Min- 
istry was planning to report on the rate of significant 
non-compliance in the hazardous waste program 
as determined through inspections, the resolution 
of exception reports over time, and the number of 
hazardous waste approvals issued over time. 

The Ministry also indicated that a jurisdictional 
review had been performed to examine the meth- 
odology used by other leading jurisdictions to 
track and report program performance measures. 
The performance measures that the Ministry was 
planning to report on are based on a combination 
of performance measures in other jurisdictions, 
the Ministry’s program and policy, and its ability to 
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track and report with the information system now 
in place. 

The Ministry told us that it had begun to prepare 
an annual report on hazardous and liquid industrial 
waste in Ontario. The report is to be based on the 
2008 HWIN data, and is to include performance 
measures approved for public release. The release 
of the first annual hazardous and liquid industrial 
waste report is scheduled for the end of 2009. Fur- 
ther to the annual report, the Spills Action Centre 
also reports on the number of spills and incidents 
relating to hazardous waste reported each year. The 
2007 Spills Summary Report is publicly available 
on the Ministry’s website. 


FINANCIAL ASSURANCE AND REVENUE 
Financial Assurance 


Recommendation 8 
To ensure that the hazardous waste operator, rather 
than the taxpayer, is responsible for financing cleanup 
costs from hazardous waste contamination, the Min- 
istry of the Environment should: 

e@ consider whether all hazardous-waste- 
management carriers and receivers should be 
required to provide financial assurance; 

e collect financial assurance prior to issuing a 
certificate of approval; and 

@ periodically review whether financial assurance 
on hand is sufficient to cover potential spills 
and future costs of cleanup, waste removal, and 
disposal. 


Status 
The Ministry reported that, to ensure that the haz- 
ardous waste operator is responsible for financing 
cleanup costs from hazardous waste contamination, 
it now requires that every receiver have financial 
assurance and that every carrier hold $1 million in 
liability insurance. 

The Ministry also indicated that it now collects 
financial assurance before a certificate of approval — 
is issued for any site that will receive, transfer, 
store, or dispose of hazardous waste. 
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To ensure that the financial assurance on hand 
is sufficient to cover the potential spills and future 
costs of cleanup, waste removal, and disposal, the 
Ministry retained a consultant to research and 
develop alternatives to the delivery of the financial 
assurance program. In order to determine the 
appropriate amount of financial assurance required 
from the receivers, in November 2007, the Ministry 
completed a guidance document that provides cur- 
rent cost information for landfill site activities. 

We were informed that, as of July 2008, all cer- 
tificates of approval for privately owned hazardous 
waste sites have a condition requiring that the 
amount of financial assurance be reassessed on an 
annual basis. The Ministry indicated that it had 
enhanced its computer system to make it a more 
effective compliance tool to track and follow up 
on financial-assurance requirements by upgrading 
the link between the certificate-of-approval and 
financial-assurance databases. The system now 
tracks whether a company has provided the appro- 
priate amount of financial assurance and identifies 
when a financial-assurance reassessment is due. 

In March 2009, the Ministry completed a new 
guidance document for waste transfer and pro- 
cessing and transfer sites. This document should 
provide ministry reviewers with the necessary 
information to calculate a more accurate financial- 
assurance amount for the site and to have a more 
efficient assessment process. 

The Ministry reviewed the certificate of approval 
for every receiver to assess the adequacy of finan- 
cial assurance for coverage of potential spills and 
future costs of cleanup and waste removal and 
disposal. The Ministry indicated that, through this 
review, it collected an additional $9.7 million in 
financial assurance. 


Hazardous Waste Fees 


Recommendation 9 
To ensure that hazardous waste fees are sufficient to 
recover program costs, are accurately recorded, and 


are collected on a timely basis, the Ministry of the 
Environment should: 

e review the objectives of the fee structure to 
ensure that the original objective of fully 
recovering program costs is still realistic and, 
if so, assess the adequacy of fees in offsetting 
program costs; 

e establish controls to ensure that the Hazardous 
Waste Information Network (HWIN) reliably 
identifies unpaid registration fees; 

@ periodically assess the reasonableness of total 
fees collected as compared to expected fees based 
on the number of registrations and manifests 
and the tonnage of hazardous waste disposals; 

@ implement procedures to ensure that all gen- 
erators certified for on-site disposal submit fees 
as required; and 

@ enhance the HWIN system so that it can calcu- 
late and identify outstanding debt by generator 
and track the amount of time debt has been out- 
standing, so that collection efforts can focus on 
generators with significant balances that have 
been outstanding for extended periods of time. 


Status 
To ensure that hazardous waste fees are sufficient 
to recover the program costs, are accurately 
recorded, and are collected on a timely basis, the 
Ministry conducted research on other jurisdictions, 
a cost study, and a study of waste generation rates, 
and, at the time of our follow-up, was reviewing 
policy options for revised fees. This review was 
expected to be completed in 2009/10 with a new 
fee structure proposed for approval and implemen- 
tation in 2010/11. | 

In response to our recommendation on estab- 
lishing controls to ensure that HWIN reliably identi- 
fies unpaid registration fees, the Ministry indicated 
that HWIN now reliably identifies unpaid registra- 
tion fees for each generator. 

The Ministry informed us that it now compares 
the total monthly outstanding receivables balance 
to the number of manifests and tonnage. The 
amounts receivable are periodically compared to 


the actual collections to calculate the collection 
rate. The reasonableness of the collection rate was 
assessed by comparison to the average collection 
rate of other government ministries. The Ministry 
also indicated that steps were being taken to calcu- 
late the HWIN collection rate on an annual basis. 

We were informed that generators that are 
certified for on-site disposal are required to report 
in accordance to the certificate-of-approval condi- 
tions. A procedure is in place to calculate and 
collect on-site disposal fees from generators at the 
time of re-registration each year. In addition, the 
Ministry may identify on-site generators through 
annual inspections. For generators that are found 
to be disposing hazardous waste on-site without 
being registered through HWIN or without a proper 
certificate of approval, compliance action is to be 
taken to ensure that they are registered in HWIN, 
have obtained necessary approvals, and have paid 
the applicable fees. 

The Ministry informed us that a new compon- 
ent had been added to HWIN to provide more 
information on outstanding receivables. The new 
component provides a breakdown of total amounts 
receivable and a report to track the amount of 
time debt has been outstanding per generator. The 
Ministry advised us that, at the time of our follow- 
up, it was validating this new reporting component 
with historical data and working with the vendor to 
ensure accuracy. 


COMPLIANCE 
Selection of Facilities for Inspection 


Recommendation 10 
To enhance the effectiveness of its inspection process, 
the Ministry of the Environment should ensure that its 
facility selection process is based on potential risk to 
the environment by: 

@ using the formalized risk-based selection process 
already developed for the district offices and 
selecting facilities for inspection based on docu- 
mented risks; 
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@ updating its risk analysis for the Sector Com- 
pliance Branch; 

@ including all potential hazardous waste gener- 
ators, carriers, and receivers in its risk assess- 
ment processes; and 

® ensuring that district and branch co-ordination 
efforts result in all high-risk facilities being 
inspected periodically. 


Status 

The Ministry informed us that facilities are now 
selected for inspection using a risk-based selection 
process, taking into consideration any potential 
risk to the environment based on their compliance 
history, approval type, volume and types of wastes, 
registration status, and exception report history in 
addition to local district knowledge. 

We were also informed that the Sector Compli- 
ance Branch had updated its risk analysis frame- 
work. The revised framework uses more up-to-date 
compliance information from the Integrated Div- 
isional System and a revised set of categories that 
are now in line with seven consequence categories 
used across the division. The Sector Compliance 
Branch also uses additional data sources such as 
HWIN to select sites for inspection. 

The Ministry informed us that it typically 
finds out about companies that have never been 
registered through incidents reported to the Spills 
Action Centre, the use of industry databases to 
identify sectors that could potentially pose high risk 
for inspection, and the comparison of the industry 
database with the HWIN database. The Ministry 
also informed us that, in developing the 2009/10 
inspection plan, it had obtained the industry data- 
base and compared it to HWIN data to identify com- 
panies that were in the industry database under 
sectors associated with hazardous waste but not 
registered with HWIN. A subset of these companies 
was assigned for inspection. 

The Investigations and Enforcement Branch also 
established an Intelligence Unit that is engaged in 
the hazardous waste sector. In 2008, it carried out 
an initiative to determine the registration status of 
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all companies that had registered in the database 
used prior to HWIN but had not registered in HWIN 
at the time of the database conversion in 2001. 

The Investigations and Enforcement Branch found 
that the majority of the companies were either no 
longer in business or were no longer required to be 
registered. We were informed that the remaining 
companies were registered and in compliance at the 
time of our follow-up. 

The Ministry informed us that, to ensure that 
district and branch co-ordination efforts result in all 
high-risk facilities being inspected periodically, the 
districts and the Sector Compliance Branch were 
co-ordinating inspection plans so that high-risk and 
underperforming facilities would be monitored. 


Inspections of Hazardous Waste Facilities 


Recommendation 11 
To ensure that inspections of hazardous waste gen- 
erators, carriers, and receivers effectively encourage 
compliance with legislation and policy, the Ministry of 
the Environment should: 
@ develop a consistent approach to rating the level 
of compliance found during its inspections; 
@ include surprise visits in its district office inspec- 
tion process; 
@ apply enforcement methods consistent with the 
degree of non-compliance; 
@ periodically verify the contents and weight of 
a sample of vehicles that transport hazardous 
waste; 
e@ implement a formal strategy for timely follow- 
up of non-compliant facilities; and 
@ review its processes to determine what other 
corrective actions to take to increase the level 
of compliance within the hazardous waste 
industry. 


Status 
The Ministry informed us that the compliance 
assessments of the Sector Compliance Branch and 
the district offices are now aligned. To ensure that 
enforcement methods are applied consistently with 


the degree of non-compliance, the Ministry put in 
place an informed-judgment matrix. The matrix 
guides the reviewer to assess the compliance level, 
and specific follow-up actions are provided for each 
compliance level. 

The Ministry also informed us that it now 
ensures that the annual inspection programs 
include both planned inspections and surprise visits 
for waste generators, carriers, and receivers. In 
2008/09, the Ministry hired additional inspectors 
to increase the ability of the district offices to con- 
duct surprise visits after normal business hours. 

The Ministry advised us that it started a periodic 
verification of contents by sampling vehicles that 
transport hazardous waste. The Ministry indicated 
that it was committed to continue vehicle sampling 
in 2009/10 to verify whether or not the contents 
being transported match the waste description pro- 
vided on the associated manifest. In 2008/09, all 
samples analyzed matched the description provided 
on the associated manifest. 

We were informed that, in May 2007, the 
Ministry implemented a formal strategy for timely 
follow-up of non-compliant facilities. It issued a 
compliance policy for applying abatement and 
enforcement tools. This policy provides guidance in 
the selection of abatement and enforcement tools to 
address violations of ministry legislation. 

The Ministry informed us that, to ensure that 
corrective actions are in place to increase the level 
of compliance within the hazardous waste industry, 
it developed a centralized exception report that 
identifies common issues of non-compliance for 
follow-up with corrective action. In addition, in 
May 2008, the Ministry distributed a letter to all 
active carriers reminding them that picking up from 
unregistered generators is a contravention of the 
regulation and their certificate of approval. The 
Ministry indicated that, although this letter served 
as a reminder, it also helped reduce the level of 
non-compliance significantly. In April 2009, a simi- 
lar letter was again distributed to all active carriers, 
along with information on the proper manifesting 
process for several types of hazardous waste. 
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Background 


Ontario’s public hospitals are generally governed by 
a board of directors that is responsible for the hos- 
pital’s operations and for determining the hospital’s 
priorities in addressing patient needs in the com- 
munity. In the 2008/09 fiscal year, the total operat- 
ing costs of Ontario’s more than 150 hospitals 

were about $22 billion ($19 billion in 2006/07), 

of which about 85% was funded by the Ministry of 
Health and Long-Term Care. 

According to the Ministry, about 844,000 
surgical procedures and 135,000 other diagnostic 
procedures (such as biopsies and imaging) were 
performed in hospital operating rooms across 
Ontario in 2006/07, at a cost of about $1.2 billion. 
This cost includes nurses’ salaries and medical sup- 
plies, but excludes most physicians’ services, such 
as surgeons’ services, which the Ministry pays for 
through the Ontario Health Insurance Plan. 

In our 2007 Annual Report, we noted that the 
Ministry had introduced several good initiatives to 
help hospitals improve surgical processes, includ- 
ing a pilot project to centralize patient referral 
and assessment, which provides patients with the 
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option of choosing a surgeon with the shortest wait 
list and assesses whether surgery is the most appro- 
priate course of action. However, the Ministry did 
not have information available on the total number 
of operating rooms in Ontario, the hours operating 
rooms were in use, the total number of patients 
waiting for surgery, or the type of surgery they were 
waiting for. 

Our audit focused on the management and use 
of surgical facilities with respect to meeting patient 
needs. We conducted work at three hospitals— 
Toronto East General Hospital, St. Joseph’s 
Healthcare Hamilton, and Sudbury Regional 
Hospital—that performed about 44,000 surgical 
procedures in their 42 operating rooms during 
the 2006/07 fiscal year. We concluded that the 
hospitals were managing the use of their surgical 
facilities well in some areas, such as implementing 
procedures to prioritize urgent surgical cases and 
screen elective patients prior to surgery. However, 
the three hospitals needed to better utilize their 
surgical facilities to reduce patient wait times. Our 
observations also included the following: 

e An average of 12% of operating rooms at the 
hospitals we visited were not used most week- 
days in 2006, and generally were not used for 
elective surgeries on weekends or statutory 
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holidays. As well, for approximately nine 
weeks in summer 2006, only about 60% of 
operating rooms were used, owing primarily 
to planned vacation-time closures. 

At the hospitals we visited, each surgeon’s 
operating room time was based primarily on 
the time allocated to that surgeon in prior 
years, rather than on other factors such as 
patients’ needs and hospital priorities. 

Most urgent emergency cases had their sur- 
gery within hospital-established time frames 
at the two hospitals we visited that tracked 
this information, although about 13% of non- 
emergency but urgent (for example, acute 
appendicitis) patients did not. 

Despite clinical guidelines indicating that 
most medically stable patients undergoing 
low-risk surgeries do not require a pre- 
operative electrocardiogram (ECG) or chest 
x-ray, research indicated that the rate of ECGs 
and chest x-rays conducted in Ontario hos- 
pitals prior to surgery varied significantly for 
patients undergoing low-risk procedures. 
None of the hospitals we visited followed up 
with the applicable surgeon—as required by 
the Ministry—to ensure that patients wait- 
ing longer than the established 10-month 
benchmark were reassessed. At one hospital, 
67% of low-priority hip-replacement patients 
waited longer than their targeted time frame 
for surgery, with some patients still not having 
had their surgery after three years. 

The timeliness of surgery varied significantly 
in some cases, depending on the hospital 

or Local Health Integration Network. For 
example, some hospitals were able to perform 
lower-priority cancer surgeries more quickly 
than other hospitals were able to perform 
more urgent cancer surgeries. 

At two of the hospitals we visited, about 

13% of the in-patient beds were occupied by 
individuals no longer requiring hospital care 
but who were waiting for alternative accom- 
modation. This reduced the number of post- 


operative beds available, sometimes resulting 
in surgical patients having their surgeries 
delayed or cancelled. 

@ The Ministry’s Provincial Infectious Diseases 
Advisory Committee indicates that “flash 
sterilization” (a quick sterilization process 
for surgical instruments) should be used only 
in emergency situations. However, we noted 
that this was not always the case, as flash 
sterilization was often used in non-emergency 
situations, such as when there was a shortage 
of instruments. 

In addition, we acknowledged that there would 
be challenges—for the hospitals, as well as for the 
Ministry and Local Health Integration Networks— 
in addressing the observations and recommenda- 
tions in our report, especially those that would 
require the co-operation of all key stakeholders, 
including fee-for-service physicians who are paid by 
the Ontario Health Insurance Plan (OHIP), not the 
hospitals. 

We made a number of recommendations for 
improvement and received commitments from 
the hospitals we visited and the Ministry that they 
would take action to address our concerns. 


Status of Recommendations 





The three hospitals we conducted work at, as well 
as the Ministry, provided us with information in 
spring and summer 2009 on the status of our rec- 
ommendations. According to this information, sig- 
nificant progress has been made in implementing 
most of the recommendations we made in our 2007 
Annual Report, although it will take several years 
for some to be fully implemented. For example, it 
will be a few years before some projects and initia- 
tives—such as those relating to reducing the num- 
ber of patients who no longer need hospital care but 
are occupying hospital beds—are completed. The 
status of the actions take by the hospitals, and the 


Ministry where applicable, is summarized following 
each recommendation. 


ACCESSING SURGERY 


Information on Operating Room Availability 
and Use 


Recommendation 1 
To better ensure the efficient use of operating rooms 
to meet patient needs, the Ministry of Health and 
Long-Term Care, in conjunction with the Local Health 
Integration Networks and hospitals, should obtain 
and review information on the number of operating 
rooms across Ontario and the extent of their use. 


Status 
The Ministry indicated that, as of March 2009, it 
was tracking the number of operating rooms for all 
hospitals through the Ontario Hospitals Reporting 
System/Management Information System. Further- 
more, the Ministry indicated that it was providing 
the Local Health Integration Networks with train- 
ing on the system and in how to interpret reports on 
hospitals’ operating room utilization. 

As well, the Surgical Efficiencies Target Pro- 
gram, a web-based tool designed to track and 
monitor predetermined performance indicators, 
including operating room availability and use, 
was implemented in all hospitals participating 
in the Ministry’s Wait Time Strategy in 2007/08. 
Participating hospitals (including the three visited 
during our audit) enter operating room data into 
this system. This has enabled them, as of March 
2009, to compare their results to provincial targets 
to identify areas for improvement. 


Allocation of Operating Room Time to 
Surgeons 


Recommendation 2 
To better ensure the most effective use of surgical 
resources and that patient needs are met in as timely a 
manner as possible, hospitals should adopt the recom- 
mendations of the Ministry of Health and Long-Term 
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Care’s Surgical Process Analysis and Improvement 
Expert Panel on allocating surgical operating room 
time to surgeons, which place more emphasis on 
patient needs than on the time that each surgeon has 
historically been allocated. 


Status 
At the time of our follow-up, the Ministry indicated 
that it would continue to work with and encourage 
hospitals to implement the recommendations of the 
Surgical Process Analysis and Improvement Expert 
Panel. 

One hospital noted that it was working toward 
allocating operating room time to surgeons on the 
basis of patient needs, and that its Operating Room 
Utilization Committee passed a motion in June 
2009 to assign all surgeons one block per week of 
operating room time and allocate all other blocks 
on the basis of demand. The Committee will meet 
again in September 2009 to further determine the 
mechanism for redistributing operating room time 
on the basis of patient demand as evidenced by 
surgical wait lists. 

Another hospital indicated that the availability 
of wait list data through the Wait Times Informa- 
tion System (WTIS) was helpful in determining 
the priority of surgeons’ needs for operating room 
time. This hospital commented that the allocation 
of operating room time to surgeons is still not a hard 
science, but that data on patient wait times, as well 
as data on operating room and surgeon utilization, 
make objective decisions easier to achieve. Although 
the hospital noted that funding is still its primary 
determinant in surgeon access to operating room 
time, funding from the Local Health Integration 
Network has enabled the hospital to make more 
operating room time available for non-cancer gen- 
eral surgeries to surgeons whose patient wait times 
are longer than established WTIS benchmarks. 

The third hospital stated that it was imple- 
menting an information management system, 
which is intended, among other things, to enable 
the hospital to measure the allocation of operating 
room time to surgeons, patient wait times, and 
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access to operating rooms on the basis of patient 
priority levels, by surgeon and service. The hospital 
expected to have the system fully implemented by 
February 2010 and planned to use information pro- 
vided by this system to modify its operating room 
schedule on the basis of patient needs and hospital 
resources, such as instrument availability. The hos- 
pital also planned to allow surgeons to keep part of 
their operating room time unbooked up to 48 hours 
prior to surgery, which would allow the surgeons 
to keep a percentage of their scheduled time avail- 
able for booking more urgent patients. The hospital 
anticipated that the results of these changes would 
provide the information required to determine the 
need for urgent time by service as well as the total 
overall distribution of operating room time by pri- 
ority, need, and service. 


Scheduling of Patients for Surgery 


Elective Surgery 

Recommendation 3 
Hospitals should periodically compare the actual time 
taken for surgeries—including operating room set-up 
and cleanup—with the time estimated for completing 
those surgeries (as indicated by the time booked for 
the operating room) and identify any recurring sig- 
nificant deviations, so that adjustments can be made 
to improve operating room utilization. 


Status 
The Ministry indicated that, since July 2007, 
hospitals participating in the wait time initiative 
have been tracking the accuracy of operating room 
scheduling by comparing the estimated duration 
of each surgery with the actual time required to 
complete the surgery. This enables hospitals to 
identify any recurring significant deviations so that 
adjustments can be made to improve operating 
room utilization. 

At the time of our follow-up, one hospital indi- 
cated that its operating room committee regularly 
reviewed information on the estimated versus 
actual time for surgeries, and adjusted operating 
room scheduling when indicated. The hospital 


noted that, although operating room utilization 
had improved, there were still some scheduling 
gaps, because adjustments are not made until 10 of 
the same surgeries show a mismatch between the 
estimated and actual time required to complete the 
surgery. 

Another hospital stated that it established a 
Surgical Utilization Committee in January 2009, 
whose terms of reference include monitoring 
whether surgeries start on time; identifying barriers 
to start times; monitoring time required to com- 
plete surgeries; and providing recommendations 
to improve operating room utilization and access. 
This hospital uses an automated procedure that 
calculates the average time taken by each surgeon 
to complete his or her last 10 cases. This average 
time is used to book the operating room but can be 
manually adjusted (for example, for a complex case 
that requires more time). 

The third hospital indicated that it reviews and 
uses data for determining the appropriate total time 
for each surgical procedure by surgeon. This infor- 
mation is discussed at its Perioperative Executive 
meetings. Once the hospital has fully implemented 
its new information management system, it antici- 
pates that it will be able to base each surgeon’s 
operating room time per surgical case on the actual 
average time it took the surgeon to complete the 
past seven similar cases. This will include time to 
set up and clean the operating room. The hospital 
anticipates that this will improve the accuracy of 
its case duration data. In addition, this hospital 
is reviewing the processes used by its scheduling 
office and will develop new policies and guidelines 
as required to enhance the accuracy of its case 
duration data. 


Emergency Surgery 
Recommendation 4 
To better ensure the equitable and timely treatment of 
patients requiring urgent surgery, hospitals should: 
@ inconjunction with the Ministry of Health and 
Long-Term Care (Ministry) and Local Health 
Integration Networks, and considering any 


recommendations from the Ministry’s Surgical 
Process Analysis and Improvement Expert. Panel, 
complete the development of and implement a 
consistent patient priority classification system 
across Ontario hospitals for emergency and 
other urgent surgical cases; 

@ review whether urgent patients are being 
prioritized by all surgeons in accordance with 
hospital policy, as well as whether these patients 
are receiving surgery within the established time 
frames, and take.corrective action where neces- 
sary; and 

@ review the costs and benefits of dedicating oper- 
ating room time each day for urgent surgical 
cases as part of their regular planned activity, 
in accordance with recommendations from 
the Ministry’s Surgical Process Analysis and 
Improvement Expert Panel. 


Status 
At the time of our follow-up, the Ministry indicated 
that a consistent patient priority classification 
system had been implemented for emergency and 
urgent surgical cases by hospitals participating in 
its Wait Time Strategy. The finalized patient priority 
codes and their definitions, as well as examples of 
procedures that fall under each code, were made 
available to all hospitals in May 2008. The over 80 
hospitals participating in the Ministry’s Wait Time 
Strategy are to use the Surgical Efficiencies Target 
Program to review whether or not patients are 
receiving care within these time frames. 

All three hospitals confirmed that they had 
adopted the Ministry’s patient priority classification 
system for emergency and urgent surgical cases, 
and indicated that they regularly review whether 
surgeons are complying with the patient prioritiza- 
tion classification system. As well, one hospital 
implemented a process to address any instances of 
surgeons who do not adhere to the classification 
system. 

The three hospitals all indicated that they mon- 
itored wait lists to ensure that patients received their 
surgery within the established time frames for each 
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priority level. One hospital commented that lower- 
priority cases would be bumped to a higher category 
if they exceeded the initially targeted wait time. 
One hospital stated that it now sets aside 
operating room time for patients requiring urgent 
access in order to meet the guidelines for max- 
imum patient waits based on patient priority. For 
example, the orthopaedic service has urgent and 
emergency access time built into its daily operating 
room time, because statistics demonstrate there 
is a constant demand for emergency orthopaedic 
surgery. Another hospital allocates operating room 
time for urgent cases on a daily basis, and the third 
hospital continues to schedule “trauma blocks” of 
operating room time. 


Pre-operative Patient Screening 
and Testing 


Recommendation 5 
To increase the efficiency and cost-effectiveness of pre- 
operative patient screening, hospitals should: 

@ establish policies, based on the patient’s needs, 
on whether the patient’s screening prior to sur- 
gery should be completed at the hospital or by 
other means, particularly for healthy, ambula- 
tory patients undergoing elective surgery; 

@ determine specifically which patients, based 
on their condition, should be required to see an 
anaesthesiologist as part of the screening pro- 
cess, rather than requiring all such patients to 
be seen by an anaesthesiologist where this is the 
current practice of the hospital; and 

@ incorporate into their screening policies guide- 
lines on pre-operative patient tests endorsed 
by the Guidelines Advisory Committee of the 
Ontario Ministry of Health and Long-Term Care 
and Ontario Medical Association. 


Status 
At the time of our follow-up, the Ministry indicated 
that it would continue to work with and encour- 
age hospitals to implement the characteristics 
of an effective pre-operative patient screening 
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program as noted by the Surgical Process Analysis 
and Improvement Expert Panel in its report. For 
instance, patients who have similar clinical condi- 
tions and are scheduled for similar procedures 
should be screened and tested in a like manner 
regardless of surgeon, anaesthesiologist, or the 
surgeon’s preferred approach to the procedure. 

In this regard, the Ministry noted that the patient 
screening process is one area reviewed by the perio- 
perative coaching teams, which were established by 
the Ministry and are made up of hospital peers with 
experience in the effective management of pre- 
operative resources. Between December 2006 and 
April 2009, the teams visited 56 hospitals. 

One hospital indicated that most pre-admission 
screening is completed at the hospital, but that it 
can also be done over the phone for patients living 
more than a two-hour drive away. As well, the hos- 
pital’s anaesthesiologists have established clinical 
criteria to indicate if a patient needs to be screened 
by an anaesthesiologist prior to surgery. The hospi- 
tal also noted that it has incorporated the screening 
policies guidelines on pre-operative patient testing 
into its practices. 

Another hospital indicated that it continues to 
follow the guidelines endorsed by the Guidelines 
Advisory Committee and that its Department of 
Anaesthesia also continues to ensure that pre- 
operative testing ordered by attending surgeons is 
performed in accordance with guidelines from the 
Canadian Society of Anaesthesiologists. 

The third hospital indicated that it compared its 
pre-operative assessment and screening with those 
of other teaching hospitals and hospitals within 
its Local Health Integration Network. The results 
showed that this hospital’s practice of having close 
to 100% of the patients screened by an anaesthe- 
siologist prior to surgery was not consistent with 
what other hospitals did. However, the hospital’s 
anaesthesiologists stated that their practice ensures 
a higher standard of care and patient safety. There- 
fore, the hospital decided not to change its current 
practice. The hospital also noted that, once it has 
fully implemented its new information manage- 


ment system (targeted for February 2010), it plans 
to work with its surgeons and anaesthesiologists to 
streamline and standardize its pre-operative testing 
requirements. 


WAIT TIMES 


Recommendation 6 
To enable both patients and health-care providers 
to make informed decisions and to help ensure that 
patients receive the surgery that meets their needs 
within an appropriate length of time, the Ministry 
of Health and Long-Term Care—in conjunction with 
Local Health Integration Networks, hospitals, and 
surgeons—should monitor patient wait times by each 
priority level and by surgeon for all types of surgery. 
As well, the Ministry should make information on 
patient wait times by priority level available to the 
public and reconsider its decision not to report at 
a future time wait times by surgeon or, as a min- 
imum, make this information available to referring 
physicians. 


Status 
At the time of our follow-up, the Ministry was col- 
lecting wait time information on certain surgical 
procedures from hospitals participating in the Wait 
Time Strategy, through its Wait Time Information 
System (WTIS). The Ministry noted that hospitals 
have the capability to generate priority-level and 
surgeon-level reports from the WTIS, and that 
public reporting on wait times by each priority level 
began in April 2008. As well, in October 2008, the 
Ministry began publicly reporting wait times for all 
general surgery, ophthalmology, and orthopaed- 
ics, which the Ministry indicated represent over 
50% of all surgeries in the province. The Ministry 
anticipated that by fall 2009, the wait times for all 
surgical procedures at hospitals participating in the 
Wait Time Strategy would be captured and reported 
publicly. The Ministry also commented that, at 
present, there are no plans to report publicly on 
wait times by surgeon or make this information 
available to referring physicians because the health 


system is moving away from surgeon-specific wait 
lists. In this regard, the Ministry anticipated that 

it would have nine pilot projects—primarily for 
joint replacements—implemented by the end of 

the 2009/10 fiscal year. These projects centralize 
patient referral and assessment, which provides 
patients with the option of choosing a surgeon 

with the shortest wait list or choosing another 
surgeon knowing what the wait time will be for that 
surgeon. 

One hospital commented that it publicly reports 
wait times for certain surgeries on its website. 
Another hospital indicated that it tracks patients 
whose waits exceed recommended times for each 
priority level by individual surgeon, and ensures 
that alternative options are provided for these 
patients, such as having the surgery performed by 
a different surgeon. The third hospital noted that it 
had made a number of changes to better manage its 
wait times. For example, patient priority classifica- 
tions and wait times are now used by the surgeon to 
schedule patients for surgery, and monthly reports 
are now provided to each surgeon on his or her wait 
times and sent to the head of each surgical specialty 
for review. 


Use of the Wait Time Information System 
by Surgeons and Hospitals 


Recommendation 7 
To monitor and manage patient wait lists more effi- 
ciently, the Ministry of Health and Long-Term Care 
and hospitals should continue to jointly develop more 
standardized reports, utilizing data from the new 
Wait Time Information System, that would readily 
provide hospitals and surgeons with useful and com- 
parative information on patient wait times. As well, 
hospitals should periodically test the accuracy of their 
key data elements in the System. 


Status 
At the time of our follow-up, the Ministry indicated 
that many standardized reports are now available 
on the Wait Time Information System (WTIS), 
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including reports on patient wait times by priority 
level and by surgeon, as well as number of surgical 
cases cancelled. In May 2008, a web-based tool was 
implemented that enables hospitals to more easily 
access this information. As well, the Ministry indi- 
cated that it had provided all hospitals with training 
on the WTIS. Furthermore, the Ministry stated that 
more standardized reports are being developed for 
the WTIS as users learn how the available informa- 
tion can be used. 

One hospital indicated that it reviews all data that 
appear to be outside of the benchmark wait times 
or where wait times are increasing to determine the 
reason and take appropriate follow-up action. Any 
inaccurate data identified are communicated to the 
Ministry’s Wait Time Information Office. Another 
hospital indicated that it verifies monthly with the 
surgeons’ offices the accuracy of its patient waiting 
list. The third hospital commented that it cross-ref- 
erences WTIS data with its operating room data on 
a daily basis, and contacts surgeons’ offices to clarify 
any discrepancies. As well, two of the hospitals noted 
that they participate annually in the Wait Time 
Information Office’s data quality validation program, 
which involves validating a sample of data elements 
to identify any data quality issues. 


OPERATING ROOM EFFICIENCY 


Monitoring of Performance Indicators 
for Operating Room Use 


Recommendation 8 
To determine if surgical resources are being utilized 
efficiently and effectively, hospitals should utilize the 
information provided by the new Surgical Efficiencies 
Target Program to monitor key performance meas- 
ures against performance targets (once the targets are 
established by the Ministry of Health and Long-Term 
Care), as well as against internal benchmarks and the 
performance of comparable hospitals. 


Status 
According to the Ministry, at the time of our follow- 
up, key performance targets and best practices had 
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been developed for the Surgical Efficiencies Target 
Program (Program). These included performance 
targets for start time accuracy and operating room 
utilization, as well as best practices for cancel- 
lations of surgery, operating room closures, and 
surgical volumes. 

One hospital noted that, at the time of our 
follow-up, it was monitoring five key items against 
the Program’s benchmarks. The hospital anticipates 
that monitoring these key indicators will allow a 
more proactive look at its challenges, as well as 
enable it to determine the changes required to 
improve throughput, reduce overtime, improve 
operating room utilization, and prevent bottle- 
necks. As well, the hospital completed a review 
of its patient flow processes in spring 2009 and is 
developing strategies to improve these processes. 

Another hospital informed us that it uses the 
information provided by the Program to monitor 
performance by operating room and surgical 
service. As well, its surgery governance council 
reviews the indicators quarterly and has policies 
to address variations. However, the hospital noted 
that there are occasional bottlenecks that lead to 
the cancellation of scheduled cases owing to lack 
of beds, especially critical care beds. Furthermore, 
additional efforts to reduce emergency room wait 
times can cause scheduled surgical cases to be can- 
celled owing to a lack of available beds. 

The third hospital compares its information 
from the Program to its peers’ in terms of perform- 
ance and benchmark thresholds, and indicated that 
results are reported monthly to its operating room 
committee. 


Surgical Bottlenecks 


Availability of Hospital Beds 

Recommendation 9 
To help ensure that patients receive the care they need 
and to reduce the cancellation of elective patient sur- 
geries, the Ministry of Health and Long-Term Care, in 
conjunction with hospitals and Local Health Integra- 
tion Networks, should develop and implement strat- 


egies to reduce the number of patients who no longer 
require hospital care but are occupying hospital beds. 


Status 
At the time of our follow-up, the Ministry stated 
that the complex issue of patients who no longer 
require hospital care but are occupying hospital 
beds needs to be addressed systematically. This 
requires the involvement of various groups such as 
hospitals and long-term-care homes. 

The Ministry indicated that it is working with 
the Local Health Integration Networks (LHINs) 
on a number of initiatives to address this issue, 
including: 

@ increasing home care and community support 

services; 

e placing additional Community Care Access 
Centre staff in hospitals to allow for faster 
access to community services for hospital 
patients; 

e funding temporary transitional beds in select 
communities for patients who are awaiting 
placement in long-term-care homes or other 
community-based settings; and 

@ providing funding for the LHINs to invest in 
local solutions to address patients requiring 
an alternative level of care. 

As well, the Ministry noted that its Ontario 
Health Performance Initiative is co-ordinating a 
quality improvement project focused on improving 
patient flow in a group of 90 hospitals. The project 
focuses on a number of areas, including improve- 
ments in the discharge planning process, to enable 
more effective and timely discharge of patients 
from hospital. The project is expected to be com- 
pleted by summer 2011. The Ministry also stated 
that it is developing a system that will, among other 
things, provide information on how long those 
patients who no longer require hospital care wait 
for access to the appropriate level of care, such 
as a long-term-care home. As well, the Ministry 
indicated that by winter 2012 it expected to have 
almost 2,000 new beds in long-term-care homes. 


All three hospitals indicated that they have 
ongoing challenges regarding patients occupying 
hospital beds who no longer require hospital care. 

One hospital stated that this issue has not yet 
been adequately addressed by its LHIN and the 
Ministry, which impacts its patients’ access to sur- 
gery, because the beds are not available for surgical 
patients. However, the hospital indicated that it was 
working extensively with its LHIN and community 
partners to address this issue. 

Another hospital stated that it is experiencing an 
increased risk of cancellation of surgeries resulting 
from the Ministry’s emergency room wait time 
strategy, which gives emergency patients preferred 
access to intensive care units. This compounds its 
persistent issue of decreased bed access that results 
from patients occupying hospital beds when they 
require an alternative level of care. Although this 
hospital indicated that there is a collaborative plan 
within its LHIN that has decreased historically long 
waits for patient placements in long-term-care 
homes, it also stated that the problem of patients 
waiting for an alternative level of care remains 
significant and is affecting access to post-operative 
care for scheduled patients. 

The third hospital also noted the challenges that 
directly related to the flow of patients through its 
emergency room, but that it had changed its daily 
bed management structure in November 2008 to 
ensure that forecasting of the need for surgical beds 
occurs more proactively, so as to prevent surgical 
cancellations. 


Availability of Anaesthesiologists 
Recommendation 10 
To help ensure the best utilization of anaesthesiology 
services, while still ensuring that patients requiring 
anaesthesia receive it in a safe and efficient manner: 
@ the Ministry of Health and Long-Term Care 
should analyze the results of the anaesthesiol- 
ogy care teams pilot projects and, if warranted, 
encourage the expansion of this concept to other 
Ontario hospitals while reviewing current fund- 
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ing mechanisms to ensure that they support this 
initiative; and 

@ hospitals, in conjunction with the College of 
Physicians and Surgeons of Ontario, should 
determine under what circumstances an anaes- 
thesiologist needs to be present for cataract 
surgeries. 


Status 
The Ministry indicated at the time of our follow-up 
that the final report from phase one of the anaes- 
thesiology care teams pilot projects was received 
in spring 2009. The Ministry anticipated that the 
report evaluation would be completed by the end of 
2009. Depending on the results of the evaluation, 
the Ministry stated that it will consider expanding 
anaesthesia care teams to other surgical areas at 
hospitals participating in its Wait Time Strategy. 

Two of the hospitals indicated that they are fully 
staffed with anaesthetists, and the third stated 
that its anaesthesia human resources has remained 
stable and has had minimal effect on its ability to 
offer surgical services. 

One hospital further commented that it is par- 
ticipating in the anaesthesiology care teams pilot 
project, which has been very successful in enabling 
the hospital to extend the availability of anaesthe- 
tists for other surgeries. However, it expressed con- 
cerns that, unless there is funding for the program 
that trains anaesthesia assistants, the expansion of 
this project would be curtailed. This hospital also 
stated that hospitals would need financial support 
to pay anaesthesia assistants, as anaesthesiologists 
are paid through OHIP, but the assistants must be 
paid directly by the hospital. 

Another hospital indicated that it uses anaesthe- 
sia assistants for cataract surgery and is expanding 
their use to support the hospital’s obstetrical 
practice and its emergency/urgent operating room 
activity, as well as for other procedures. 

The third hospital noted that its practice is to 
assign an anaesthetist for all cataract procedures 
unless there is not one readily available. 
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No additional work has been completed by the 
hospitals or the Ministry, such as in conjunction 
with the College of Physicians and Surgeons of 
Ontario, to determine under what circumstances an 
anaesthesiologist needs to be present for cataract 
surgeries. 


SURGICAL INSTRUMENTS 


Recommendation 11 
To better ensure that cleaned and sterilized surgical 
instruments are available when needed for surgeries, 
hospitals should: 

e in light of the Provincial Infectious Diseases 
Advisory Committee’s (PIDAC’s) best practices 
guidance, re-examine the practice of using flash 
sterilization in non-emergency situations; 

@ where flash sterilization is used, ensure that a 
record is maintained of the instruments that 
are flash sterilized, including the name of the 
surgeon who subsequently used the instrument 
and the name of the patient it was used on, in 
accordance with PIDAC’s recommendations; and 

e review the costs and benefits of implementing an 
instrument-management system to track instru- 
ment location and status. 


Status 
In November 2007, the Ministry, in conjunction 
with the Ontario Hospital Association, forwarded 
a letter to all hospitals asking them to review their 
sterilization procedures in relation to the Provincial 
Infectious Diseases Advisory Committee (PIDAC) 
guidelines. Later that month, the Ontario Hospital 
Association convened a videoconference to discuss 
the issue of flash sterilization with hospitals, which 
PIDAC chaired. Furthermore, in January 2008, the 
Ontario Hospital Association distributed to all hos- 
pitals a two-page fact sheet on flash sterilization, 
which was developed in association with PIDAC 
and also posted on the Ministry’s website. The fact 
sheet provided further guidance to hospitals on 
when it is acceptable to use flash sterilization and 
on the information that must be documented when 
it is used. 


At the time of our follow-up, one hospital noted 
that it had undertaken an extensive replacement 
of instruments over the last two years and that 
duplicate instruments have been purchased to 
prevent the need to flash sterilize one-of-a-kind 
instruments. The hospital also indicated that flash 
sterilization is now used sparingly, and that all 
instances of use are recorded and tracked, and the 
data retained in accordance with PIDAC’s recom- 
mendations. Further, the hospital stated that 
monthly audits are completed to ensure compliance 
with PIDAC’s recommendations. The hospital also 
included an instrument tracking system in its cap- 
ital plan for 2010. 

Another hospital stated that it now meets 
PIDAC’s recommendations on the use of flash steril- 
ization. As well, this hospital indicated that it mon- 
itors its use of flash sterilization and has reduced its 
use by 31% from 2007/08 to 2008/09. The hospital 
also noted that new surgical equipment have been 
purchased, with a primary focus on reducing flash 
sterilization. As well, the hospital stated that its 
new instrument management system will be imple- 
mented by February 2010. 

The third hospital indicated that it is using flash 
sterilization strictly in accordance with PIDAC 
guidelines. To do this, the hospital implemented 
a policy that reflects the PIDAC standard for the 
use of flash sterilization, purchased more instru- 
ments, improved its documentation, and improved 
its instrument-tracking methods to consistently 
monitor flash sterilization. The hospital stated 
that all instances of flash sterilization are reviewed 
monthly, and any use outside the PIDAC guidelines 
is addressed. Furthermore, the hospital noted that 
since December 2008, it averaged only four instan- 
ces per month of flash sterilization and that they 
were in accordance with PIDAC-approved reasons. 
With respect to an instrument management system, 
the hospital indicated that it was investigating the 
cost/benefit of outsourcing instrument reprocess- 
ing with a company that has a system that tracks 
the surgical instruments’ tray location and instru- 
ment status. 


Chapter 4 
Section 






Background 


Long-term-care homes, such as for-profit and not- 
for-profit nursing homes and charitable homes, 
provide care, services, and accommodation to indi- 
viduals unable to live independently and requiring 
the availability of 24-hour care and supervision in 
a secure setting. There are more than 600 homes 
in Ontario caring for about 75,000 residents, most 
of whom are 65 or older. In the 2008/09 fiscal 
year, funding by the Ministry of Health and Long- 
Term Care (Ministry) to long-term-care homes 
totalled $2.8 billion ($2.8 billion in 2006/07), with 
residents generally also making a co-payment of 
between $1,600 and $2,200 (between $1,500 and 
$2,100 in 2006/07) per month for accommodation. 
Residents of long-term-care homes usually have 
conditions requiring treatment with medication 
prescribed by a doctor. According to the Ministry, 
in the 2008/09 fiscal year it paid pharmacies about 
$359 million ($333 million in 2006/07) for more 
than 25 million drug prescriptions (19 million in 
2006/07) dispensed for residents of long-term-care 
homes. As well, the Ministry’s Ontario Govern- 
ment Pharmaceutical and Medical Supply Service 
provides certain drugs, such as acetaminophen 
(generic Tylenol), at no charge to long-term-care 
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homes. In 2008/09, the cost of such drugs was 
about $5.2 million ($3.4 million in 2006/07). 

In our 2007 Annual Report, we assessed whether 
medications for residents were managed in an effi- 
cient, safe, and controlled manner, in accordance 
with applicable legislation and required policies 
and procedures. Medication management involves 
physicians, pharmacists, and nurses contracted 
by the homes (see Figure 1). Our audit indicated 
that at all of the three long-term-care homes we 
visited—Hamilton Continuing Care in Hamilton, 
Leisure-world St. George in Toronto, and Provi- 
dence Manor in Kingston—there were a number 
of procedures in place to ensure that the homes 
obtained physician-prescribed medications and 
administered them to residents in a safe and timely 
manner. However, we noted areas where these 
homes could improve their medication manage- 
ment practices. Some of our more significant obser- 
vations included the following: 

e At all three homes, documentation to indicate 
that informed consent was obtained from 
residents or their substitute decision-makers 
for the use of new medications was either 
nonexistent or inadequate. 

e Two of the homes we visited were not doing 
an adequate job of reporting all medication 
errors, and during 2006 reported only 12 and 
26 errors respectively. The identification and 
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Figure 1: Medication Management—Professional Responsibilities 
Source of data: Ministry of Health and Long-Term Care 


Physicians: Prescribe medications for long-term-care home residents and review the resident’s care plan—including 
medications—on the basis of the physician’s knowledge and skill and the clinical situation of an individual resident. Physicians 
are accountable to their regulatory body, the College of Physicians and Surgeons of Ontario. 


Pharmacists: Dispense medications for long-term-care home residents on the basis of physicians’ or other recognized health 
professionals’ prescriptions and the pharmacists knowledge of the resident and the prescribed drug, in accordance with 
provincial and federal legislation as well as in accordance with the standards of practice of their regulatory body, the Ontario 
College of Pharmacists. 


Nurses: Apply their knowledge of the resident and the medication when assessing residents, administering medications, 
evaluating residents’ reaction to medications, and planning and documenting the medication administration process, as per 
the Medication Practice Standard of the College of Nurses of Ontario. Nurses act as the liaison between the physician and 
pharmacist in relation to medication management for each resident, and collaborate with the health-care team in the long-term- 
care home to maintain safe medication-management processes. 


review of medication errors is important in 
preventing similar errors in the future. 

We obtained and analyzed information on 
drugs dispensed to residents of all long-term- 
care homes through the Ministry’s Ontario 


level-1 alert, we believe some follow-up may 
be warranted given the high percentage of 
alert overrides. 

None of the three homes we visited periodic- 
ally reconciled controlled substances admin- 


Drug Benefit Program. On the basis of this istered to residents with records of drugs 


analysis, we noted that, during 2006, more received from the pharmacy and those on 
hand. 


care homes were dispensed at least one e Processes to ensure that medications 


than 5,700 residents of Ontario long-term- 
of eight high-risk drugs that international approaching their expiry date—including 
experts have concluded are generally more 
harmful than beneficial to older adults. As 

well, at least 20% of residents in 30 homes 


those in emergency supplies—are identified 
and removed from use upon expiry needed to 
be strengthened. 
were dispensed these drugs. While we e Two of the homes did not consistently use 
acknowledge that medications are generally environmentally responsible practices to dis- 
prescribed by physicians, we believe there pose of unneeded medications. 
may be situations where a high rate of use of We made a number of recommendations for 
such higher-risk drugs in certain homes may improvement and received commitments from the 
warrant some follow-up by the Ministry in three long-term-care homes we visited and the 
conjunction with the College of Physicians Ministry that they would take action to address our 
and Surgeons of Ontario. 
@ Ninety-one percent of the 18,000 level-1 


alerts (which warn of a drug combination 


concerns. 


that is clearly contraindicated and should not 


Status of Recommendations 





be dispensed or administered) generated by 
pharmacy computers were overridden and the 
drugs dispensed to residents of 421 long-term- 
care homes. While pharmacists may have 


In spring and summer 2009, the long-term-care 
homes, as well as the Ministry of Health and Long- 
contacted the prescribing physician to discuss Term Care, where applicable, provided us with 


these drug interactions prior to overriding the information on the status of our 2007 Annual Report 


recommendations. According to this information, 
the long-term-care homes had made substantial 
progress on several of our recommendations and at 
least some progress on most others. Although the 
Ministry had taken some action on the recommen- 
dations directed to it, in a few areas, the Ministry 
was waiting for recommendations from the Joint 
Task Force on Medication Management it convened 
in May 2008 before deciding what action to take. 
The recommendations were expected by the end 

of summer 2009. The status of the action taken on 
each of our recommendations at the time of our 
follow-up is as follows. 


PROVISION OF MEDICATIONS 


Recommendation 1 
To help promote the safe and efficient provision of 
medication to residents, long-term-care homes should 
ensure that: 

@ contracts with pharmacies specify the type and 
frequency of procedures the pharmacy is to 
perform, as well as the reporting methods to be 
used, with respect to assessing the home’s com- 
pliance with medication-related policies; and 

@ consent to treatment with new medication is 
obtained and documented from either the resi- 
dent, when capable of giving consent, or from 
the resident’s substitute decision-maker in a 
timely manner. 

The Ministry of Health and Long-Term Care should 
review its policy on standing orders (which typically 
relate to over-the-counter medication) to determine if 
additional guidance is necessary. 

As well, to help promote the health of residents, 
long-term-care homes, in conjunction with the Min- 
istry of Health and Long-Term Care, should develop 
a consistent definition of what constitutes a medica- 
tion error. In addition, long-term-care homes should 
ensure that medication errors are consistently identi- 
fied, documented, and reviewed so that appropriate 
action can be taken on a timely basis to minimize 
similar occurrences in the future. 
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Status ; 
At the time of our follow-up, one home had signed 
a new pharmacy contract when it changed its phar- 
macy provider in October 2007. Although the new 
contract now requires the pharmacy to conduct 
regular audits to review and monitor medication 
storage, administration, and documentation at the 
home, it does not specify the type and frequency 
of procedures the pharmacy was to perform or 
the reporting methods to be used. Another home 
indicated that, instead of revising its pharmacy 
contract, it had developed with its contracted 
pharmacy an annual plan of activities the pharmacy 
would perform. The third home commented that 
it was developing a reporting tool to consistently 
monitor the home’s compliance with medication- 
related policies and procedures. It anticipated 
completing the tool in November 2009 and planned 
to include specific reporting requirements related to 
this in its next pharmacy contract. 

With respect to consent to treatment, one home 
indicated that it had conducted staff training on 
the subject and had implemented periodic reviews 
of resident charts to ensure that consent is being 
obtained and documented for new medications 
ordered. Another home stated that it had held 
discussions on the role of its physicians and staff 
in obtaining informed consent, which resulted in it 
reviewing and updating its policy and procedures 
for processing physician orders for medications. 
The home also revised its procedures so that medi- 
cation order sheets receive a stamp within which 
nurses are to document that the resident or the resi- 
dent’s substitute decision-maker had been notified 
of a medication change. The third home developed 
a policy and procedure to ensure that consent for 
new medications is received in a timely manner. It 
also included on its record of physician orders for 
medications a place where staff are to indicate that 
the resident or his or her family has been notified 
about the treatment change. 

At the time of our follow-up, all three homes had 
policies that contained a definition of what consti- 
tutes a medication error, although these policies 
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continued to vary among the homes. As well, all 
of the homes had policies to document and review 
medication errors. For example, one home indi- 
cated its quality council reviews medication errors 
monthly and its professional advisory committee 
reviews these quarterly. This home also noted that 
it takes corrective measures, such as staff educa- 
tion, if its policy on reporting medication errors is 
not being followed. 

In May 2008, the Ministry convened a Joint 
Task Force on Medication Management including 
physicians and representatives from long-term- 
care homes and pharmacies. At the time of our 
follow-up, the Ministry informed us that the task 
force was examining larger system issues—which 
would not specifically include standing orders 
and medical directives—and would be making 
recommendations pertaining to the appropriate 
management of medications from a system perspec- 
tive. The Ministry also indicated that the task force 
had examined the definition of medication errors 
used by selected institutions and jurisdictions, and 
had reviewed the definition of medication errors 
that are reportable on the Ministry’s new Critical 
Incident System, which had been implemented in 
all homes by April 2008. The Ministry told us that 
the task force might have some recommendations 
on medication errors in its final report, which was 
expected by summer 2009. 


REACTIONS TO MEDICATIONS 


Recommendation 2 
To help reduce the risk of adverse medication reac- 
tions in residents, long-term-care homes should: 

® ensure that residents more likely to experience 
adverse reactions—those taking a new higher- 
risk medication, for example—are monitored 
more closely than other residents and that 
results of this monitoring are documented; 

e develop and implement policies to ensure 
consistent identification and documentation 
of adverse drug reactions, so that action can be 
taken to prevent future occurrences; and 


e adopt consistent criteria for referring residents 
to specialized psychogeriatric programs and 
ensure that sufficient staff are appropriately 
trained in those criteria. 

In addition, the Ministry of Health and Long-Term 
Care, in collaboration with the College of Physicians 
and Surgeons of Ontario (CPSO), should periodically 
review the use of higher-risk drugs at long-term-care 
homes, as well as the frequency with which residents 
receive drugs with unique drug-to-drug interaction 
alerts, or alternatively provide access to this informa- 
tion to the CPSO and other appropriate regulatory 
bodies so that appropriate follow-up action can be 
taken where the use of higher-risk drugs and the fre- 
quency of pharmacist overrides of alerts seem unduly 
high. 


Status 
At the time of our follow-up, one home indicated 
that it had developed a psychotropic medication 
form to monitor the effect of new or changed doses 
of such medications on residents. According to this 
home, it documents this monitoring for seven days, 
during which time it would expect to identify any 
adverse reactions to medications. Another home 
stated its contracted pharmacy sends the home 
monthly a list of all residents receiving high-risk 
drugs. The home revises the applicable residents’ 
care plans as necessary to identify risks and the 
interventions necessary to minimize potential 
adverse effects. The home also noted that its pro- 
fessional advisory committee reviewed high-risk 
medications in April 2009 and was developing a 
long-term plan to further address this issue. As 
well, this home indicated that, in January 2007, its 
contracted pharmacy began sending an adverse- 
drug-interaction alert along with a medication if 
the pharmacy was concerned about a potential 
adverse reaction. The resident’s physician is to 
review this form, which is placed in front of the 
resident’s medication-administration record so that 
staff giving medications are aware of the risk. The 
third home noted that its.contracted pharmacist 
provides recommendations to physicians regarding 


the use of high-risk medications and alternatives 

to their use. The home monitors more closely 
residents who take new high-risk medications and 
notes the effect of the medication in the resident’s 
file. As well, the contracted pharmacist performs a 
medication screening quarterly and may make rec- 
ommendations to the home’s physicians on matters 
such as drug dose and drug interactions. 

One home noted that, in October 2007, it 
implemented a policy on preventing and detecting 
adverse consequences of medications. Another 
home had revised its policy on readily identifying 
problems associated with newly marketed drugs. 
The third home also had policies in place to address 
adverse events. This home told us that it not only 
documents any adverse drug reactions residents 
have while in the home, but also any a new resi- 
dent may have had prior to admission. The home 
also stated that it records adverse reactions in its 
incident-management system and notifies the con- 
tracted pharmacy. 

At the time of our follow-up, one home had 
implemented a procedure for referring residents 
to specialized psychogeriatric services. It also indi- 
cated that it was continuing to offer opportunities 
for its registered staff to participate in specialized 
training provided by psychogeriatric consultants. 
Another home noted that two staff members were 
trained in a psychogeriatric program and work 
with residents who require a specialized program. 
One registered staff member accompanies the 
psychiatrist when he or she sees residents and fol- 
lows up on his or her recommendations. The home 
also indicated that every second month it conducts 
“mini mentals”—a short cognitive status test—with 
residents who have dementia. The third home’s 
revised policy indicated that its nursing manage- 
ment team was to discuss incidents of unacceptable 
resident behaviour and, when necessary, make 
referrals to the psychogeriatric resource person and 
psychogreriatric outreach team. 

At the time of our follow-up, the Ministry 
indicated it expected all long-term-care homes to 
implement a common assessment tool by 2010. This 
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tool is to provide homes with drug-related quality 
indicators—such as residents taking nine or more 
different medications—which will enable care pro- 
viders to identify for increased monitoring residents 
with a higher risk of having adverse medication 
effects. The tool is also expected to generate reports 
on the impact of certain medications. The Ministry 
also indicated that over 400 homes had registered 
to complete the Medication Safety Self-Assessment 
for Long-Term Care developed by the Institute 

for Safe Medication Practices Canada. One of the 
homes we audited noted that it had completed the 
self-assessment and, at the time of our follow-up, 
was using the results to further improve its medica- 
tion management practices. 

With respect to periodically reviewing the use of 
higher-risk drugs in collaboration with the College 
of Physicians and Surgeons of Ontario, the Ministry 
indicated that it intended to share information 
with the appropriate stakeholders once it received 
the recommendations of the Joint Task Force on 
Medication Management, expected by end of 
summer 2009. 


SAFEGUARDING MEDICATIONS 


Recommendation 3 
To better safeguard medications against possible theft 
or accidental misuse, long-term-care homes should: 
@ ensure that staff access to drugs is limited as 
much as practicable, and in accordance with 
legislation and standards, regardless of where 
the medications are stored; and 
@ periodically reconcile records of drugs admin- 
istered with those received and on hand for 
narcotics and other drugs that may be more 
susceptible to theft (such as benzodiazepines), 
and take immediate follow-up action if the rec- 
onciliations indicate unaccounted-for narcotics. 


Status 
At the time of our follow-up, one home had imple- 
mented a policy stating that registered staff, such 
as nurses, could only pass their keys, which grant 
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access to medications, to other registered staff. Keys 
were not to be given to unregistered staff members, 
such as personal support workers. This home also 
told us that it has only two sets of keys available to 
access the medication carts, which further limits 
the number of staff with access. Another home 
noted that, at the time of our follow-up, all of its 
medication carts required a key to open them, 
rather than having a combination keypad. The 
third home stated that its managers periodically 
verify that medication carts and storage rooms are 
kept locked when not in use and follow up with the 
applicable nurse if a problem is noted. 
Furthermore, one home noted that, com- 
mencing in June 2009, its contracted pharmacy 
was performing a reconciliation of the narcotics 
administered with those received and on hand for 
one resident each month. Another home indicated 
that it has implemented a reconciliation process for 
all benzodiazepines so that these are recorded on 
a benzodiazepine count sheet and counted at shift 
change along with the narcotics. The third home 
stated that it decided not to revise its narcotics 
policy and that it does not periodically reconcile 
records of drugs administered with drugs received 
and on hand. It added, however, that its contracted 
pharmacy completes biannual audits of narcotics. 


EXPIRED MEDICATIONS 


Recommendation 4 

To help ensure that residents receive safe and effective 
medications, long-term-care homes should implement 
processes to ensure that medications approaching 
expiry are identified and removed from use upon 
expiry. | 

In addition, to ensure that adequate (but not 
excessive) levels of medications are available when 
needed, long-term-care homes should establish min- 
imum reorder levels and maximum order quantities 
for medications in the emergency drug stock and for 
medications supplied by the Ontario Government 
Pharmaceutical and Medical Supply Service in 
accordance with resident usage. 


Status 
At the time of our follow-up, one home indicated 
that its pharmacy was conducting monthly reviews 
of its emergency drug stock, bi-monthly reviews of 
medications supplied by the Ontario Government 
Pharmaceutical and Medical Supply Service, and 
quarterly reviews of all other medications at the 
home to identify and remove ones approaching 


_ expiry. As well, the director of care was monitoring 


the home’s emergency drug stock and medications 
supplied by the Ontario Government Pharmaceut- 
ical and Medical Supply Service. Another home 
revised its policies to require staff to check the 
expiry date on medications prior to administering 
them. The third home indicated that it trained staff 
on placing “date opened” stickers on all eye drops, 
insulin, and other medications, as required. It also 
told us that its pharmacy monitors its emergency 
drug stock and replaces medications due to expire. 
Furthermore, the home noted that it was con- 
ducting weekly reviews of its drug carts and medi- 
cation rooms to ensure the removal of any expired 
medications. It also indicated that its pharmacy 
conducts monthly reviews, which, the home noted, 
had identified no expired medications as of the time 
of our follow-up. 

To help ensure that adequate (but not excessive) 
levels of medications are available for residents, 
one home established minimum and maximum 
amounts for all emergency drug stock medications 
and medications supplied by the Ontario Govern- 
ment Pharmaceutical and Medical Supply Service. 
Another home was using the minimum quantities 
that could be reordered as established by the 
Ontario Government Pharmaceutical and Medical 
Supply Service and had established amounts of 
each drug that should be in its emergency drug 
stock. It was doing this instead of establishing min- 
imum reorder levels and maximum order quantities 
in accordance with resident usage. The third home 
told us that it was conducting periodic audits of 
supply orders to ensure that excessive amounts of 
drugs were not being ordered. It noted that it had 
established maximum quantities to have on hand 


of government pharmacy supplies. As well, this 
home indicated that it was reviewing its inventory 
of government pharmacy supplies before ordering 
additional medications, which it noted would also 
help ensure that there were no thefts. It had also 
established amounts of each drug that should be in 
its emergency drug stock, and was reordering drugs 
to bring the quantity on hand to these levels. 


DESTRUCTION OF EXCESS MEDICATION 


Recommendation 5 
To help minimize medication waste and potential 
misappropriation, as well as to promote the efficient 
and environmentally responsible disposal of excess 
medication, long-term-care homes should: 

@ in conjunction with the Ministry of Health and 
Long-Term Care and the Local Health Integra- 
tion Networks, review ways to streamline the 
drug-tracking and -destruction process while 
retaining sufficient safeguards over this process; 
and 

@ periodically monitor staff to ensure that they 
are following accepted policies for disposing of 
expired and excess medication. 

While developing regulations for Bill 140 (the new 
act on long-term-care homes), the Ministry of Health 
and Long-Term Care should also consider the feas- 
ibility of alternatives such as those used in other juris- 
dictions with respect to the destruction of unopened 
packaged medications that are still usable. 


Status 
At the time of our follow-up, one home indicated 
that it was continuing to use a contracted service 
to ensure the efficient and environmentally respon- 
sible disposal of excess medications. As well, this 
home noted that it was supporting ongoing learn- 
ing opportunities for registered staff regarding 
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safe medication practices, and that such practices 
were included in the performance management of 
registered staff. Another home stated that it had 
made no changes to assess the extent to which 
medications are wasted. However, the home 
indicated that it has held meetings with staff and 
managers responsible for medication management 
to remind them of the accepted policies for dispos- 
ing of expired and excess medications, and it has 
informed its health and safety committee members 
of these policies. The third home noted that it had 
streamlined its drug-tracking and -destruction 
process and had trained staff on this process. It 
had also held staff education on environmentally 
friendly processes for the destruction of narcotics. 
At the time of our follow-up, the Ministry 
indicated that, through its compliance program, it 
was ensuring that medication disposal was being 
conducted according to the standards and policies 
set out in the legislation and regulations that gov- 
ern long-term-care homes. The Ministry noted that 
it had consulted its Joint Task Force on Medication 
Management while developing the regulations for 
Bill 140. The Ministry posted the first draft regula- 
tion for comment in May 2009 and expected to 
post a second regulation later in the year, which 
is to cover the area of medication management. 
However, the Ministry told us that, although it 
had considered options such as those used in 
other jurisdictions with respect to the destruction 
of unopened packaged medications that are still 
usable, it had concluded that the unopened pack- 
aged medications returned from long-term-care 
homes should be destroyed. It stated this is because 
the World Health Organization’s guidelines indicate 
that only drugs that have not been previously dis- 
pensed are considered acceptable for donation. 
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Section 


Registry 


After the brutal 1988 murder of 11-year-old Chris- 
topher Stephenson by a convicted pedophile, the 
Legislature enacted Christopher’s Law, 2000 (Act). 
The Act established the Ontario Sex Offender 
Registry (Registry) in 2001 to track the where- 
abouts of individuals living in Ontario but convicted 
anywhere in Canada of one or more designated sex- 
ual offences. The Act also applied to every offender 
residing in Ontario still serving a sentence for such 
offences at the time the Act came into force. The 
Ministry of Community and Correctional Services 
(Ministry) and the Ontario Provincial Police (OPP) 
developed the Registry, and the Ministry’s Sex 
Offender Registry Unit within the OPP continues to 
administer it. The OPP and more than 140 munici- 
pal and First Nations police services are responsible 
for registering and monitoring offenders. There 
were 7,400 registered offenders as of January 2007. 
At the time of our follow-up, there were 9,142 
active registered sex offenders. The federal govern- 
ment also operates a similar registry. 

In our 2007 Annual Report, we concluded that 
while the Ministry and the OPP had worked dili- 
gently and cost-effectively to create the Registry to 
help police investigate sexual crimes and monitor 
sex offenders in their communities, the Registry 
was not yet functioning adequately to serve its 
intended purpose. Among our specific concerns, we 
noted the following: 
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e The Act establishing the Registry requires 


police services to register offenders only after 
they have completed jail or prison sentences. 
Thus, the many offenders who live in the com- 
munity while serving their sentence, or those 
awaiting appeal decisions, are not required to 
register. 

The Registry was incomplete for a number of 
reasons. We identified 365 provincial offend- 
ers who should have been registered but were 
not. As well, there was no process for regis- 
tering young offenders who received adult 
sentences. 

The Ministry never obtained a list of the more 
than 1,000 sex offenders in federal custody 

in Ontario at the time of the Registry’s incep- 
tion so that they could be registered on their 
release. In addition, there was no reliable 
reporting mechanism to ensure that all 
offenders living in Ontario were registered on 
release from federal correctional facilities. We 
identified 360 offenders released from federal 
custody who should have been registered 

but were not because of missing information 
about the timing of their release. 

There were no ministry guidelines for follow- 
ing up on non-compliant offenders—those 
who did not register, or who failed to re- 
register annually—and practices varied at 


local police services. Warrants were not con- 
sistently issued for offenders in breach of the 
Act for extended periods. While the overall 
rate of compliance was high, the rate of non- 
compliance varied widely across the province. 

e@ The search tools available in the registry 

application required improvement. Police 
investigators could not, for example, filter 
data by gender or age of victim, relation- 
ship (Gif any) between the victim and the 
offender, or the location of past crimes. In 
addition, other offender information, such as 
photographs, employment and educational 
addresses, or detailed case information was 
not always captured in the database, thus 
impairing the Registry’s usefulness. 

We made a number of recommendations for 
improvement and received commitments from the 
Ministry that it would take action to address our 
concerns. 

We also acknowledge the work of the Legisla- 
ture’s Standing Committee on Public Accounts, 
which held a formal hearing on this audit and 
issued a report in February 2009. 


Status of Recommendations 





On the basis of information provided by the 
Ministry, we concluded that the Ministry and the 
OPP have made some progress on almost all of our 
recommendations, with significant progress being 
made on most of them. We noted that Ontario’s 
Internal Audit Services also undertook an assess- 
ment of the actions taken to address our 2007 rec- 
ommendations. In December 2008, Internal Audit 
Services concluded that the Ministry and the OPP 
had made significant progress in implementing our 
recommendations. 

The status of action taken on each of our recom- 
mendations is as follows. 
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Recommendation 1 j 
To help ensure that all convicted sex offenders are 
registered, the Ministry of Community Safety and 
Correctional Services should: 

@ work with correctional and police services 
to ensure that the notice of duty to register 
is served to all convicted sex offenders at the 
appropriate times; 

® consider revising existing legislative require- 
ments to ensure that all offenders released from 
institutions and living in the community must 
register; 

@ work closely with provincial justice and correc- 
tional systems to obtain all offender records on 
an ongoing basis; 

@ work with the Correctional Service of Canada to 
obtain data on all offenders in federal custody in 
Ontario since the Registry’s inception; and 

@ consider establishing procedures to identify 
offenders moving into Ontario, and confirming 
that those who report moving out of the prov- 
ince have actually done so. 


Status 
In our 2007 Annual Report, we noted that some 
400 offenders who had completed their custodial 
sentences or who had been released on parole were 
never issued a notice of duty to register (NDR) as 
a sex offender. At that time, there was no legisla- 
tive requirement for offenders to receive NDRs. 
Although most registered of their own accord 
anyway, 17% did not. On December 5, 2008, Chris- 
topher’s Law, 2000 (Act) was amended to, among 
other things, require that the NDRs be provided. 
The Ministry informed us that it has since worked 
with correctional and parole officers on this matter. 
Correctional institutions now serve NDRs to all 
applicable sentenced inmates, and probation and 
parole officers serve NDRs to applicable probation- 
ary or paroled offenders. As well, local police ser- 
vices are now required to notify a person charged 
with a sexual offence of his or her reporting obliga- 
tions under the Act, and to forward a completed 
NDR to the Registry once an offender is convicted. 


Chapter 4 Follow-up Section 4.11 





416 J 2009 Annual Report of the Office of the Auditor General of Ontario 





b | 
= 
+ 
i= 
oS 
=] 
{*] 
® 
wn 
a. 
= 
= 
a 
x) 
be 
e 
+ 
fe 
oO 
pe 
2. 
cs 
=} 
o 


Other amendments to the Act include the 
following: 

e Sex offenders serving intermittent sentences 
(on weekends, for example), those released 
on bail pending an appeal for a sexual offence, 
and those who change their name must regis- 
ter within 15 days of being sentenced, of being 
released on bail, or of changing their name. 

@ Police services must notify the Sex Offender 
Registry Unit within the OPP immediately if 
they receive notification from a mental-health 
facility of the unsupervised release of a person 
found not criminally responsible for a sexual 
offence by reason of a mental disorder. 

e Correctional facilities must notify the Registry 
24 hours in advance of the release of any sex 
offender on an unescorted temporary absence 
pass. 

e Youth custody facilities must provide notice 
of any unescorted leaves granted to young 
persons serving adult sentences for a sexual 
offence. 

Regulations under the Act were also amended to 

require: | 

e all registered offenders to provide police with 
their driver’s licence number, along with the 
licence plate number, model, year, descrip- 
tion, and colour of any vehicle they own, 
lease, or regularly drive; and 

@ all those convicted of voyeurism under the 
Criminal Code to register. 

The Ministry further informed us that it had 
updated its policing standards and guidelines to 
reflect these changes, and conducted province-wide 
training between October and December 2008. 

The Ministry also advised us that the OPP has 
obtained from Correctional Service of Canada 
(CSC) data on all offenders in federal custody in 
Ontario since the Registry’s inception. 

With respect to our recommendation to consider 
establishing procedures to identify offenders mov- 
ing into Ontario, and to confirm that those who say 
they are moving out of the province actually leave, 
the Ministry informed us that as part of its regula- 


tory amendments, all registered offenders who 
report that they are moving out of Ontario now 
must provide detailed information about their new 
place of residence. There is still no procedure in 
place to help identify offenders from other jurisdic- 
tions who move into Ontario. We were informed, 
however, that informal business practices do exist 
whereby other provinces advise the Registry of 
offenders who are known to be moving to Ontario. 


Recommendation 2 
To ensure that all offender records are deleted only 
for legitimate reasons, the Ontario Provincial Police 
should: 

e work with the National Parole Board to obtain 
updates on pardon revocations and ensure that 
such offenders re-register on a timely basis; and 

@ track and maintain supporting documentation 
for all deletion requests. 


Status 
The Ministry informed us that it now has an agree- 
ment with the National Parole Board for the Board 
to provide notice to the Registry of any Ontario sex 
offender who has either been granted a pardon or 
has had a pardon revoked. A further agreement 
with the CSC allows the OPP access to relevant data 
from federal databases maintained by the CSC, 
the National Parole Board, and the Department of 
National Defence. 

With respect to registry deletions, the Ministry 
informed us that all offender record deletions are 
now reviewed by management. The appropriate- 
ness of the deletion is verified, and a documented 
procedure is in place to ensure that supporting 
documentation is maintained for all deletions. 


Recommendation 3 
To ensure that Registry records are maintained 
accurately, the Ministry of Community Safety and 
Correctional Services should: 
e consider eliminating the right of non-compliant 
offenders to opt out of the annual mail 
reminder; and 


e establish procedures for police services to ensure 
that reminders returned as undeliverable are 
followed up on a timely basis. 


Status 
The Ministry informed us that it has changed 
its registration process so that offenders may no 
longer opt out of receiving the annual registration 
reminder letter. To address some offender concerns 
about receiving these letters at their home address, 
the OPP directive to police services says the envel- 
ope should not include any information identifying 
the sender as police or indicate in any way that 
it relates to the Sex Offender Registry. Offenders 
can also register a separate mailing address for the 
annual reminder letter. 

The Ministry further reported that it has 
established required procedures to deal with any 
reminder letters returned as undeliverable, which 
may indicate that the offender has moved without 
reporting to police. The procedures, which have 
been distributed to all local police services, require 
them to initiate a non-compliance investigation, and 
the Sex Offender Registry Unit offers them support, 
if required. 


Recommendation 4 
To ensure that non-compliant offenders are followed 
up on ina timely manner, the Ministry of Community 
Safety and Correctional Services should: 

e develop guidelines and procedures for police 
services regarding follow-ups on offenders in 
non-compliance, including policies on the issu- 
ing of warrants; 

e work with those local police services having a 
high rate of offender registration non-compli- 
ance in their community; and 

@ consider expanding the inspection scope of the 
Public Safety Division to include registry-related 
activities. 


Status 
The Ministry informed us that it now has a guide- 
line in place for dealing with non-compliant offend- 
ers and the issuing of warrants. A training package 
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was developed and distributed in November 2008 
to ensure that police services are fully aware of the 
guideline. In addition, the Sex Offender Registry 
Unit of the OPP requires that all police services pro- 
vide a monthly report on actions taken with regard 
to non-compliant offenders. The Ministry’s Quality 
Assurance Unit, which inspects local police servi- 
ces, also receives a quarterly compliance summary 
report. We were further informed that the OPP 

has worked with those police services that had the 
highest non-compliance rates and, as a result, the 
number of non-compliant offenders was reduced 
from 490 as of September 2007 to 303 as of Sep- 
tember 2008. In our 2007 audit, we noted that the 
overall compliance rate stood at 95%; the Ministry 
reported that as of January 2009, the overall com- 
pliance rate was over 97%. 

With respect to our recommendation to consider 
expanding the scope of the Public Safety Division’s 
inspections to include registry-related activities, we 
were informed that the Division will do this in its 
next cycle of quality assessment inspections, set to 
begin in 2011. 


Recommendation 5 
To help improve the Registry’s usefulness for quickly 
identifying potential suspects in an investigation, the 
Ontario Provincial Police should: 

e create the ability to search or filter data by vic- 
tim gender, victim age, relationship (if any) to 
the offender, and the location of past offences; 

@ consider expanding the collection of other useful 
offender information, such as vehicle informa- 
tion and family-contact data; 

e ensure that police verify offender information in 
a timely manner; and 
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e reinforce the requirement for all offenders to pro- 





vide a residential street address when registering. 


Status 
The Ministry informed us that it has upgraded the 
Registry’s system software to enable police to filter 
data in several new ways to quicken the search pro- 
cess. These upgrades include the ability to filter data 
by sex of victim, age of victim, relationship (if any) 
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between victim and offender, offender age, hair col- 
our, skin tone, and the location of past sex offences. 

We were further informed that other system 
enhancements were made to facilitate the new 
requirements for: 

e information about offender driver licences 

and vehicles, including licence plate numbers; 

@ police services to input address verification 

particulars and to identify and track unveri- 
fied addresses; and 

@ expanded case narrative information on what 

the offender did, including a list of designated 
sex acts, to assist investigative specialists in 
identifying more quickly persons of interest. 

Police services are now responsible for making 
reasonable efforts to verify a sex offender’s address 
at least once a year. Offenders may no longer 
provide a post office box instead of a residential 
address, although they may continue to use a post 
office box as a mailing address. 

The Ministry also informed us that the interface 
between its Offender Tracking Information System 
(OTIS) and the Registry has been improved to 
allow for more data to be transferred automatic- 
ally between the two systems. As well, the OTIS 
system itself can now house more information 
and communicate more readily with correctional 
facilities and parole offices to identify more quickly 
offenders being released into the community. The 
OPP and the Ministry’s technology unit continue to 
work on creating an interface between the Ontario 
and national sex offender registries to enable the 
electronic transfer of data between the two. 


Recommendation 6 
To help improve the usefulness and accountability of 
the Registry, the Ontario Provincial Police should: 

@ ensure that sufficient training and support are 
provided to local police services; 

@ prioritize outstanding system-change requests 
and devotelsufficient resources to address them 
in a timely manner; 

e correct all known system-report errors to ensure 
that police have access to accurate information 


when accessing the registry database for investi- 
gative purposes; and 

e ensure that all funds approved for registry pur- 
poses are actually spent on registry activities. 


Status 
The Ministry informed us that province-wide 
training for all police services regarding the 
legislative and other revisions to the Registry was 
completed in December 2008. Further training was 
provided in October 2009. In addition, the OPP 
was developing an e-learning program for remote 
training of police services. In addition, we were 
informed that, in February 2009, the Ontario Asso- 
ciation of Chiefs of Police sponsored a workshop 
on the recent changes to the Registry, with over 
50 representatives from various police services 
attending. A second workshop was planned for the 
2009/10 fiscal year. 

The Ministry informed us that the OPP and 
the Ministry’s Technology Services Group worked 
together to prioritize the outstanding system- 
change requests, and the highest priority requests 
were implemented between September and 
December 2008. A second group of priorities was 
implemented in October 2009. 

With respect to the system-report errors, the 
Ministry informed us that these have been cor- 
rected, except for two being addressed as part of 
a wider justice technology upgrade scheduled for 
completion during the 2009/10 fiscal year. 

The Ministry further reported that the OPP is 
now allocating to the Registry all ministry funding 
earmarked for the Registry. The Sex Offender Regis- 
try Unit’s increased expenditures for each of the last 
two fiscal years reflect these budget allocations, and 
the Ministry reported that the additional funding 
has helped address the gaps in training and the sys- 
tem errors we noted in our 2007 Annual Report. 


Recommendation 7 
To ensure that the Registry is always available to 
the police, the Ministry of Community Safety and 
Correctional Services should complete the Registry’s 


disaster recovery plan and test its effectiveness as soon 
as possible. 


Status 
The Ministry informed us that the OPP has com- 
pleted a disaster recovery plan and successfully 


tested its effectiveness in March 2008 and again in 
March 2009. 


Recommendation 8 
To help ensure that confidential information in the 
Registry is adequately protected from unauthorized 
access and modification, the Ministry of Community 
Safety and Correctional Services should: 

@ ensure that the Ontario Provincial Police’s 
security reviews are performed regularly in 
accordance with policy and that recommenda- 
tions arising from these reviews are imple- 
mented on a timely basis; and 

e regularly review system-access rights to ensure 
that information in the Registry is available 
to users strictly on a need-to-know basis and 
that authorization to make database changes is 
strictly controlled. 


Status 

The Ministry informed us that in October 2008, the 
OPP hired an external firm to conduct a compliance 
review of its Public Key Infrastructure (PKI) secur- 
ity system, designed to ensure that all information 
transmitted to and from the Registry is encrypted to 
prevent unauthorized access. The firm concluded 
that the OPP was in compliance with the require- 
ments for PKI certification and all other applicable 
policies and procedures. 

With respect to system-access rights, the OPP 
further reported that it had developed a process 
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whereby the registry supervisor in each police ser- 
vice is now responsible for monitoring and updating 
local user access rights. The new process was first 
tested at one local police service in October 2008, 
and an associated policy document was completed 
the following month and communicated to all police 
services during training sessions provided in the 
final months of 2008. The new process is scheduled 
for province-wide implementation in 2009/10 on 
completion of related system enhancements, which 
will provide local supervisors with on-line access to 
a listing of users and their roles at their local police 
service. Further guidance on the revised process will 
be provided to all police services at the time of the 
province-wide implementation. 


Recommendation 9 
To demonstrate the effectiveness of resources dedi- 
cated to the Registry, the Ministry of Community 
Safety and Correctional Services’ Public Safety 
Division should work to develop appropriate perform- 
ance measures for the Registry, including evidence 
that it is proving helpful to police in the resolution of 
sexual-crime investigations. 


Status 
The OPP informed us that it has worked with the 
Ministry’s Policy and Strategic Planning Division to 
develop a set of comprehensive performance meas- 
ures for the Registry, and a draft of the proposed 
measures was provided to us. These measures were 
awaiting ministry approval. In addition, we were 
provided with a number of case-history summaries 
where the Registry data was instrumental in resolv- 
ing police investigations. 
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Section 





Background | 


After sustained human-to-human transmission 

of the H1N1 influenza-A virus, the World Health 
Organization declared on June 11, 2009, that the 
first worldwide influenza pandemic in 41 years— 
and the first of the new century—was under 

way. The Ministry of Health and Long-Term Care 
(Ministry) is responsible for formulating emergency 
plans for infectious-disease outbreaks such as 
influenza pandemics. In the 2008/09 fiscal year, 
the Ministry spent about $44 million to ensure that 
Ontario will be prepared in the event of such an 
outbreak. 

We reported on our audit of the Ministry’s out- 
break preparedness and management in our 2007 
Annual Report that a number of measures had been 
taken since the SARS outbreak of 2003 to improve 
the province’s readiness to respond to outbreaks 
of infectious disease. However, we also noted that 
Ontario, like many other jurisdictions, still was not 
adequately prepared to respond to large-scale out- 
breaks of infectious disease. In particular, we noted 
the following: 

e The Ministry had no assurance, despite 

the comprehensive response plan it had 
developed, that everyone in the health system 
knew what to do in planning for and during 
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a pandemic. One-third of the public health 
units had not completed their local pandemic 
plans and some health-care stakeholders were 
unsure who should be responsible for stock- 
piling critical supplies. 

Designers of the critical-care triage tool 
included in the Ministry's pandemic plan 
recommended that the tool be tested and 
submitted for public consultation but neither 
was ever done. The tool is intended to help 
physicians in acute-care settings make the dif- 
ficult decisions about prioritizing critical care 
during a pandemic. 

The availability of non-hospital sites where a 
significant number of people could be quar- 
antined or isolated for an extended time was 
limited. The Ministry had no plans to look for 
additional isolation sites for future outbreaks 
despite its experience during the SARS out- 
break, when it was unable to find suitable 
venues. 

Although the Ministry instructed local public 
health units in 2006 to establish up to 750 
temporary influenza-assessment centres 

to relieve pressure on hospitals and other 
primary-care providers, this had generally not 
been done when we completed our audit in 
mid-2007. 


e There were a significant number of public 
health staffing vacancies, with approximately 
one-third of public health units being without 
full-time medical officers of health. In addi- 
tion, close to 100 public health positions 
within the Ministry were vacant, including 
some designated as critical during a human- 
health emergency. 

e@ We found there were no warehouses for the 
storage of pandemic supplies west of Toronto. 
This means the Toronto warehouse, with a 
capacity equal to the combined capacity of 
the two warehouses in Northern Ontario, 
would have to serve a population about eight 
times the size of that served by the Northern 
Ontario facilities. In addition, there had been 
no formal assessment of the potential risk 
involved in storing all supplies for Southern 
Ontario in a single location. 

@ The Ministry could not reach some health- 
care providers because it had been told that 
contact information held by the College of 
Physicians and Surgeons of Ontario could 
be used only in emergencies. Consequently, 
the Ministry had to purchase the information 
from a third party—but the data so acquired 
was incomplete. 

We made a number of recommendations for 
improvement and received commitments from the 
Ministry that it would take action to address our 
concerns. 


Status of Recommendations 


We concluded that the Ministry has, based on the 
information it provided to us, made good progress 
in addressing most of our recommendations, with 
additional work continuing on others due to the 
longer-term nature of certain of our recommenda- 
tions. Further work was needed in such areas as 
ensuring all local public-health units have the 
necessary plans and resources in place, addressing 
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staffing shortages in the Ministry’s Public Health 
Division, and ensuring there are sufficient infection- 
control practitioners available in non-hospital 
settings. 

The status of action taken on each of our recom- 
mendations is described below. 


PLANNING AND CO-ORDINATION 


Recommendation 1 
To ensure a consistent and co-ordinated response to 
infectious-disease outbreaks across the province, the 
Ministry of Health and Long-Term Care should: 

@ review both the Ontario Health Plan for an 
Influenza Pandemic (OHPIP) and the Ministry 
Emergency Response Plan regularly to update 
these documents as necessary; 

@ translate the OHPIP into French as required by 
legislation; 

@ as recommended by the World Health Organiza- 
tion, periodically conduct simulation exercises 
to confirm that its response plan on infectious- 
disease outbreak will work properly; 

e clarify the responsibilities of all relevant parties 
so that all parties understand their responsibil- 
ities—for example, by providing a summary 
or checklist of planning activities by pandemic 
phase and by organization in the next version of 
the OHPIP; and 

e develop a template to help public health units 
complete local pandemic plans. 


Status 

The Ministry advised us that it had updated the 
Ontario Health Plan for an Influenza Pandemic 
(OHPIP) each year, with release of the 2009 edi- 
tion set for fall 2009. Although the Ministry last 
updated its Ministry Emergency Response Plan in 
2007, it intended to conduct a review of this plan in 
fall 2009 to incorporate lessons learned from emer- 
gency exercises conducted in 2008 and the H1N1 
influenza-A virus outbreak in spring 2009. 

The Ministry informed us that it had received an 
exemption from translating the OHPIP into French 
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because of its technical and scientific content. 
However, a number of OHPIP summary-level docu- 
ments were translated into French and published 
on the ministry website. As well, the Ministry 

in August 2008 invited agencies designated and 
identified under the French Language Services Act to 
join an advisory group to provide advice on ways 
to further integrate French-language resources into 
pandemic-planning activities. 

Since our last audit, the Ministry told us, it had 
led and participated in a number of exercises to 
test the OHPIP and other features of its pandemic- 
preparedness programs. These included a mass 
immunization exercise in 2007 and a government- 
wide full-scale pandemic exercise in 2008. The 
Ministry was working on preparing lessons-learned 
reports from these activities at the time of our 
follow-up. 

The Ministry also advised us that it had clarified, 
documented, and summarized roles and respon- 
sibilities for various parties, including elected 
officials in the government of Ontario, the Chief 
Medical Officer of Health, municipalities, and most 
health-service providers. The Ministry was in the 
process of further clarifying and summarizing roles 
and responsibilities for community health-service 
providers and primary-care practitioners for inclu- 
sion in the 2009 edition of OHPIP. 

With respect to developing a template to help 
those public health units that had not drawn up 
their local pandemic plans, the Ministry told us it 
had pursued other ways to assist the units. At the 
time of our follow-up, the Ministry indicated that 
six of the 10 public health units that did not have 
a finalized plan in place were on their way to com- 
pleting a plan. The remaining four faced challenges 
associated with community engagement, and indi- 
cated that a template would not be of significant 
use in resolving these issues. The Ministry reported 
that it would work with the public health units over 
summer 2009 to assess their level of pandemic pre- 
paredness for a potential H1N1 response. 


HEALTH-SYSTEM RESOURCES 
Acute Care in Hospitals 


Recommendation 2 
To ensure that access to acute care in an outbreak ts 
fair and equitable to all Ontarians, the Ministry of 
Health and Long-Term Care should: 

e consider the need for public consultation, 
particularly since its recently developed critical- 
care triage tool may be the first one developed 
anywhere in the world; 

e work closely with the medical community to test 
and refine the critical-care triage tool; and 

e establish a plan for responding to various levels 
of surges in patients needing critical care. 


Status 
In its initial response to this recommendation, 
the Ministry said it had launched a pilot study in 
February 2007 to test the best method for gauging 
the tool’s efficacy and accuracy, and was expecting 
results by March 2008. However, the Ministry 
advised us during our follow-up that this study 
had been delayed due to difficulties in conducting 
statistical analysis on the generated data. The 
research team continued to work on the analysis 
and expected to have results completed by winter 
2009, at which time they would be shared with the 
Ministry. In the meantime, the Ministry advised us 
that two consultation sessions covering how Ontar- 
ians perceived the critical-care triage tool were held 
in March 2008. The Ministry informed us that it 
would address the feedback obtained from these 
consultation sessions in its work on communication 
planning. 

In an effort to refine the critical-care triage tool 
and to address unique circumstances of the pediat- 
ric community, medical experts had devised a sep- 
arate Pediatric Critical Care Triage Protocol, which 
was included in the 2008 version of the OHPIP. The 
Ministry informed us that it would look to conduct 
further testing and consultation on both the adult 
and pediatric tools by summer 2010. 


The Ministry reported that a project to integrate 
and co-ordinate approaches to manage surges in 
outbreaks was tested and introduced to all hospitals 
in March 2009, with implementation slated for 
spring 2010. Subsequently, the Ministry planned to 
take part in a simulation exercise in January 2010 
with participating hospitals and Local Health Inte- 
gration Networks to test the efficacy of the program 
in responding to large-scale outbreaks. In addition, 
the Ministry had provided more resources in the 
OHPIP to guide the acute-care sector in developing 
a phased approach to surge-capacity management 
during an influenza pandemic. 


Isolation and Quarantine; Transfer of 
Patients with Infectious Diseases; and 
Influenza Assessment, Treatment, and 
Referral Centres 


Recommendation 3 
To ease the burden on hospitals during an infectious- 
disease outbreak, the Ministry of Health and Long- 
Term Care should: 

@ ensure that local public health units identify 
suitable non-hospital quarantine sites for indi- 
viduals not requiring hospital care and deter- 
mine if they are properly equipped or how they 
are to be equipped, so that they will be available 
when they are needed; 

@ give due consideration to making participation 
in the Provincial Transfer Authorization Centre 
compulsory to help prevent the spread of infec- 
tious diseases between facilities; 

@ resolve the legal, licensing, scope-of-practice, 
and funding aspects of community-based influ- 
enza assessment, treatment, and referral cen- 
tres, and monitor their establishment by public 
health units; and 

e make alternative arrangements in advance if 
it is likely that certain local public health units 
will not have established the required assess- 


ment centres. 
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Status 
The Ontario Agency for Health Protection and 
Promotion, a new arm’s-length agency of the gov- 
ernment, advised the Ministry in April 2009 that 
although there may be a need for quarantine in 
outbreaks of certain known or unknown diseases, 
there is no such need in an influenza pandemic. 
The Agency recommended that, to facilitate care 
for infected individuals who don’t require hospital 
admission but who lack access to home isolation, 
and to mitigate community spread of illness during 


an infectious-disease outbreak, the Ministry should: 


@ consider non-hospital treatment sites for 
symptomatic individuals who don’t require 
hospital care; 

e develop criteria for these sites, including 
location, capacity, equipment, supplies, and 
recommended staffing; and 

@ develop guidelines and a template agreement 
that could be used by local public health units 
in their negotiations with owners of potential 
treatment sites to facilitate procurement and 
optimize consistency of this process. 

The Ministry indicated it needed to further 

examine the Agency’s recommendations. 

While participation in the Provincial Transfer 
Authorization Centre had not yet been made com- 
pulsory for health-care facilities at the time of our 
follow-up, the Ministry advised us that data from 
the Centre as of January 2009 indicated substantial 
participation and compliance rates. In any case, the 
Ministry said, the Chief Medical Officer of Health 
has the statutory authority to order health-care 
providers to use the Centre in the event of an 
outbreak of immediate risk to the health of people 
anywhere in Ontario. The Ministry indicated to us 
that it would further consider making participation 
in the Centre compulsory as statutory and program 
opportunities arose. 

The Ministry informed us that the resolution 
of many legal, licensing, scope-of-practice, and 
funding issues of community-based influenza 
assessment, treatment, and referral centres would 
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occur only when an influenza pandemic strikes 
Ontario. Consistent with our recommendation, 
the Standing Committee on Public Accounts also 
recommended that the Ministry monitor public 
health units’ establishment of these centres. At the 
time of our follow-up, the Ministry was continuing 
to identify processes required in advance of open- 
ing such centres, and to monitor progress made in 
the development of local pandemic co-ordination 
plans, including preparations for referral centres. 

The Ministry advised us that it had in 2008 
revised its strategy on assessment centres, mak- 
ing available a range of alternative arrangements 
for local planners to consider when preparing for 
influenza assessment, treatment, and referral, 
which might reduce the need for these centres. We 
were told that,the Ministry would include further 
resources, such as tools and guidelines, for these 
alternative arrangements in the 2009 edition of the 
OHPIP: 


Human Resources in Public Health, 
and Human Resources in the 
Health-care Sector 


Recommendation 4 
To enhance the availability of human resources dur- 


‘ ing an infectious-disease outbreak, the Ministry of 


Health and Long-Term Care should: 

e take effective measures to fill the large number 
of vacancies of medical officers of health in the 
public health units and of other positions in the 
Ministry’s Public Health Division and public 
health laboratories; 

@ inconjunction with professional associations 
and regulatory colleges, maintain up-to-date 
registries of volunteer health-care providers who 
would be available to assist during outbreaks; 
and 

@ monitor the success of local public health units 
in recruiting health-care retirees and other 
volunteers who could help in an outbreak 
situation. 


Status 
At the time of our 2007Annual Report, we reported 
that one-third of public health units were without 
a full-time medical officer of health (MOH). By the 
time of our follow-up, the Ministry informed us that 
22 of Ontario’s 36 public health units had full-time 
MOHs while the remaining 14 employed physicians 
as acting MOHs. The Ministry reported it had taken 
a number of steps to help fill the MOH vacancies, 
including: 

e allocating more funds for additional infection- 
prevention and -control practitioners to sup- 
port MOHs; 

@ supporting the Physician Re-entry Program, 
which enables doctors to obtain the educa- 
tional requirements for MOH positions; 

e funding the MOH-in-Training Program, 
which helps acting MOHs obtain MOH 
qualifications; 

@ raising the salaries of MOHs and associate 
MOHs through the Physician Services Agree- 
ment, effective April 1, 2009; and 

@® ensuring that boards of health appoint acting 
MOHs to provide coverage and appropriate 
services in public health units where the MOH 
position is vacant. 

In addition, the Ministry advised us that the 
Ontario Public Health Laboratories are now part of 
the Ontario Agency for Health Protection and Pro- 
motion, the new arm’s-length government agency 
mentioned earlier. As of April 2009, the Laborator- 
ies had recruited 14 specialized staff, including 
microbiologists, scientists, epidemiologists, and a 
dedicated outbreak response co-ordinator. None- 
theless, the Ministry noted that it was still address- 
ing Public Health Division vacancies at the time of 
our follow-up and could not fill all vacancies in the 
short-term. 

The Ministry said it had determined there 
would be significant challenges in maintaining 
registries of volunteer health-care providers avail- 
able to assist during outbreaks. Instead, it had 
worked with the Federation of Health Regulatory 


Colleges to determine a potential role for the 
health regulatory colleges in notifying health 
professionals in an emergency and disseminating 
requests for assistance. In addition, the Ministry 
had identified newly created resources, such as 
infection-control resource teams of the Ontario 
Agency for Health Protection and Promotion, which 
could be deployed by the Chief Medical Officer of 
Health to provide such on-site assistance as inves- 
tigating and managing an outbreak of hospital- 
acquired infections. As well, the medical colleges 

of Quebec and Ontario reached agreement in 2009 
to recognize the professional qualifications of each 
other’s physicians and to allow certain physicians to 
practise on both sides of the Quebec—Ontario bot- 
der. Canada’s health ministers also signed a memo- 
randum of understanding in 2009 on the provision 
of mutual aid in the form of health resources during 
a public health emergency. 

The Ministry indicated that in 2008 it had 
assessed the current status of local efforts in 
recruiting health-care retirees and other volunteers 
who could help in an outbreak, and learned that 
approximately half of the public health units had 
initiated this type of planning. The Ministry noted 
that it would work with the remaining public health 
units to improve overall preparedness. 


MEDICAL INTERVENTION 


Recommendation 5 
To ensure that vaccines, antiviral drugs, medical 
supplies, and personal protective equipment for 
health-care workers can be made available in suffi- 
client quantities and on a timely basis, the Ministry of 
Health and Long-Term Care should: 

e store, distribute, monitor, and administer anti- 
virals, vaccines, and personal protective equip- 
ment so that they are accessible to people when 
needed; and 

e@ emphasize to the broader health-care sector 
the importance of local stockpiling of personal 
protective equipment. 
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It should also ensure that it recovers the money 
owed to it by the federal government for its share of 
the cost of the national antiviral stockpile. 


Status 
The Ministry said it had taken some steps, and was 
working on others, to develop strategies to ensure 
antivirals, vaccines, and personal protective equip- 
ment were accessible to people when needed. 

In the case of antivirals, the Ministry established 

a working group to formulate a strategy to ensure 
antivirals are accessible to the health system within 
12 to 24 hours of when patients require treatment 
(antiviral therapy is ineffective in most patients if 
administered more than 48 hours after the onset of 
symptoms). 

In the case of vaccines, the Ministry cited the fol- 

lowing initiatives since our 2007 Annual Report: 

e@ The Ministry enumerated and mapped work- 
ers in the province’s critical infrastructure 
sectors for the purposes of health-emergency 
preparedness. 

@ While the federal government would co- 
ordinate security arrangements for the inter- 
provincial transport of vaccines, the Ministry 
would work through existing structures in the 
Provincial Emergency Operations Centre to 
marshal provincial and local security services 
to ensure safe transport of vaccines within 
Ontario. 

e Ina 2007 exercise, the Ministry analyzed the 
effect of a pandemic on the warehousing and 
distribution capabilities of the government 
pharmacy. The exercise found that the gov- 
ernment pharmacy was able to significantly 
reduce its delivery time to public health 
units and to identify contingencies needed to 
address the impact of a pandemic. 

e To address the risk of a general breakdown 
of public order at dispensing sites during 
mass vaccinations, public health units had 
developed plans that include security meas- 
ures to ensure public order. Typically, these 
plans included roles for local police or other 
security-service providers. The 2007 exercise 
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cited above also included a scenario involving 
a breakdown of public order. 

e As Ontario is currently the only province with 
a universal program for seasonal influenza 
immunization, the Ministry possesses proven 
experience in planning and delivering mass 
immunization campaigns. The established 
immunization program mechanisms also 
include policies and procedures necessary to 
track adverse reactions. 

With respect to personal protective equipment, 
the Ministry had at the time of our follow-up 
obtained almost all of the required quantities of 
medical supplies and equipment, including N95 
respirators, and expected to have all required items 
stockpiled by fall 2009. To that end, the Ministry 
informed us that it was finalizing a distribution 
strategy for the deployment of such supplies 
and equipment from the provincial stockpile to 
health-service providers across the province. Such 
a strategy would include a streamlined order and 
entry system for health-service providers to access 
needed supplies and equipment, as well as appro- 
priate audit and monitoring controls. 

To emphasize to the broader health-care sector 
the importance of local stockpiling of personal 
protective equipment, the Ministry advised us that 
it had taken several measures, including: 

@ asection in the OHPIP that communicates an 

expectation regarding such stockpiles; 

@ presentations to health stakeholders; and 

@ amonthly newsletter from the Ministry on 

emergency preparedness. 

As of April 2009, the Ministry had also provided 
the health and broader public sectors with access to 
preferred pricing agreements for personal protect- 
ive equipment and infection control supplies. This 
extends the competitive prices negotiated by the 
Ministry to the health and broader public sectors to 
encourage stockpiling of these products. 

The Ministry also informed us that it had 
recovered the money owed by the federal govern- 
ment for its share of the cost of the national anti- 
viral stockpile in 2008. 


SITUATION MONITORING AND 
ASSESSMENT 


Recommendation 6 
To allow efficient and effective disease surveillance at 
the provincial level so that the extent and seriousness 
of any outbreaks can be analyzed and the most appro- 
priate action can be taken, the Ministry of Health and 
Long-Term Care should: 

@ expedite its setting of standards for the timely 
reporting of diseases and for the completeness 
and integrity of disease data that public health 
units enter in the integrated Public Health Infor- 
mation System; and 

e make plans to ensure that any new surveillance 
system is implemented only after proper quality 
assurance—such as improving the accuracy and 
completeness of the disease data in the existing 
system before conversion—and after sufficient 
consultation with and training for users. 


Status 
We were informed that the Ministry had imple- 
mented two policies since our 2007 Annual Report 
related to the timeliness of data entry for both 
routine surveillance and urgent case reporting. 
According to the Ministry, these policies had as of 
April 2009 reduced average time between initial 
case notification at the public health unit, and entry 
into the disease surveillance system, as follows: 

@ to 11 days from the two to three weeks 
reported in our 2007 audit for routine surveil- 
lance; and 

@ to two days from 10 for urgent cases. 

The Ministry also indicated that it had taken a 
number of steps since 2007 to address completeness 
and integrity of data. These included: 

@ production of disease-specific user guides, 

which outlined the standardized definition 
of each data field, and specified system- 
mandatory fields, along with those required 
for surveillance purposes; 

@ carrying out data-cleaning initiatives that 
required health units to enter key information 


that was missing to ensure availability of com- 
plete and high quality data; and 
e sharing outbreak-data reports with public 

health units weekly, resulting in a better qual- 

ity of outbreak data entered into the disease- 

surveillance system. 

The Ministry advised us that implementation of 

a new surveillance system was expected to begin 
in spring 2009 for immunization and inventory 
management, while the modules for communicable 
disease and outbreak management were to be com- 
pleted in winter 2011, pending funding approval. 
The Ministry had established a quality assurance 
team that works with various government partners 
and public heath units to standardize processes 
and data within the new system. As the system 
progresses through implementation, national and 
international vocabulary standards for data quality 
will be implemented. A training working group was 
also established to identify training requirements, 
such as adherence to standards, protocols, and 
surveillance and reporting requirements contained 
in the Ontario Public Health Standards. The Min- 
istry also established another working group, with 
representatives from public health units and the 
Ministry, to advise on implementation of the new 
surveillance system. 


PREVENTION AND REDUCTION IN 
TRANSMISSION 


Recommendation 7 
To help minimize the public’s exposure during a dis- 
ease outbreak, the Ministry of Health and Long-Term 
Care should: 

e collect and analyze data on the sufficiency of 
infection-control resources in all health-care 
settings; 

e establish standards for the infection-control 
resources required in all health-care settings 
and follow up to ensure that these standards are 
being complied with; and 
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@ finalize the protocols for surveillance and 
management of infectious diseases at the public 
health units. 


Status 
In our 2007 Annual Report, we noted that the 
Ministry had no data on, and sometimes no stan- 
dards for, the quantity of infection-prevention and 
infection-control resources, including people and 
materials. At the time of our follow-up, we con- 
cluded that the Ministry had made improvements 
in this area but that further work was required. The 
Ministry advised us that: 

@ Since our 2007 audit, the Provincial Infectious 
Diseases Advisory Committee (PIDAC) had 
released Best Practices for Infection Preven- 
tion and Control Programs in Ontario. This 
document applies to a range of health-care 
settings, including long-term-care facilities, 
and identifies best practices on the use of 
infection-control resources. As well, Regional 
Infection Control Networks, which existed at 
the time of our audit, and Infection Control 
Resource Teams, which were recently created 
with the Ontario Agency for Health Protection 
and Promotion, supplement these infection- 
control materials and resources. 

e@ With respect to human infection-control 
resources, the Ministry had collected and 
analyzed the ratio of infection control prac- 
titioners (ICPs) to beds in the acute-care 
sector, but not other health-care settings. The 
Ministry advised us that the PIDAC best prac- 
tices document above outlined that minimum 
recommendations for staffing should not be 
based exclusively on bed numbers. The ratio 
of ICPs will vary according to the health-care 
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setting, and to the volume and complexity of 
the ICP’s work. Accordingly, while standards 
for ICP resources in the acute sector and long- 
term-care homes were available, there were 
still no standards that indicated the number 
of ICPs needed to support other health-care 
services. In addition, while the Ministry 
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concluded that, as of September 2008, the 
ICP-to-beds ratio for hospitals was at 1:100, 
which surpassed the 1:115 national standard, 
it did not record the actual ICP-to-beds ratio 
for long-term-care homes (the standard was 
12050015 250)5 

e The Ministry sets eligibility criteria for, and 
funds the recruitment of, infection control 
practitioners (ICPs). Successful ICP recruits 
who don’t have the appropriate certification 
commit to obtain it within three years of 
beginning employment. In 2007, we noted 
that 30% of ICPs in the acute-care sector were 
certified. At the time of our follow-up, the 
Ministry informed us that it did not record the 
professional designations of ICPs in the prov- 
ince, expecting that to be done by the agencies 
employing them. 

e Although compliance to new standards and 
indicators for hospitals on hand hygiene and 
patient safety was reported publicly as of 
April 2009, the hand-hygiene program would 
not be rolled out to long-term-care home set- 
tings until fall 2009. 

In the area of finalizing the protocols for surveil- 
lance and management of infectious diseases at the 
public heath units, the Ministry advised us that the 
Ontario Public Health Standards and incorporated 
protocols were released in October 2008 and came 
into effect in January 2009. To support applica- 
tion of these standards and protocols, the Ministry 
informed us that it was working on developing best 
practices and an infectious-diseases manual with 
evidence-based information on surveillance, case 
and contact management, and other public health 
interventions. These are expected to be released in 
one to two years. 


COMMUNICATION 


Recommendation 8 
To help ensure timely and coherent information- 
sharing at various stages of a disease outbreak, the 
Ministry of Health and Long-Term Care should test its 


public communication strategy with all members of 
the health-care system and the media. 


Status 
In our 2007 Annual Report, we noted that the 
Ministry had established a steering committee to 
provide supervision and advice in the redesign and 
implementation of a new public health call system 
just prior to the end of our audit fieldwork, and that 
we would review the status of this matter in 2009. 
The Ministry advised us that, as a result of the 
steering committee’s work, it had reconfigured the 
public health on-call system for after-hours cover- 
age. The system utilizes existing managers of key 
branches within the Ministry’s public health div- 
ision to support the Public Health Call Centre and 
Chief Medical Officer of Health in addressing sig- 
nificant public-health events that occur at night or 
on weekends. Due to ongoing restructuring of the 
public health division, the Ministry expected to see 
a redesign of the current call system in winter 2009. 

The Ministry indicated that it had utilized 

and tested the Health Emergency Information 
Cycle teleconference system during the provincial 
response to the new H1N1 influenza virus out- 
break. In addition, the Ministry had tested, prac- 
tised, and implemented communication strategies 
in response to a number of past actual public health 
events, including listeriosis, tuberculosis, E-coli, and 
others. Further, the Ministry organized and hosted 
a risk communication workshop in March 2009 
with medical officers of health and public health 
unit communication leads to share expertise and 
best practices as they relate to crisis communica- 
tions for public health events. 


PERFORMANCE REPORTING 


Recommendation 9 
To help enhance its ability to report publicly on out- 
break preparedness and management in a transpar- 
ent and timely manner, the Ministry of Health and 
Long-Term Care should: 


e collect data and establish reasonable bench- 
marks for relevant performance measures of 
outbreak preparedness and management activ- 
ities; and 

@ report regularly to the public on these perform- 
ance indicators. 


Status 
The Ministry told us that, since our 2007 Annual 
Report, it had taken a number of initiatives to 
enhance its ability to report publicly on outbreak 
preparedness and management. Among them: 
e It planned to release an initial report of public 
health in Ontario in summer 2009 that exam- 
ined all of the province’s public health units 
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using more than 30 indicators to facilitate 
understanding of progress by public health 
units of similar profile. 

In 2007 and 2008, the Ministry made public 
its compliance results in the area of emer- 
gency management concerning human health, 
disease, and epidemics under its obligations 
as set out in the Emergency Management and 
Civil Protection Act and the related Order-in- 
Council. 

All Ontario hospitals publicly reported 
patient-safety indicators and standards as of 
April 2009. 
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Chapter “15 Ministry of Revenue 
Section 


Wu &) Retail Sales Tax Program 


Follow-up on VFM Section 3.13, 2007 Annual Report 


e The audit selection process was suffering from 

Backgrou nd several deficiencies, including the following: 

j e The auditable tax roll used for selecting 

= oo a vendors for audit excluded many vendors 
The Ministry of Revenue (Ministry) administers ; : 
: te registered in Ontario, such as those vendors 
the Retail Sales Tax Act, which imposes a general ; 
awe registered for less than two years and those 

sales tax of 8% on the retail price of most goods 


‘ ; ; ; that designated themselves to be part-time. 
and services sold to final consumers in Ontario. As 


of March 31, 2009, approximately 427,000 vend- 
ors (420,000 in 2007) were registered to collect 
and remit retail sales tax (RST) to the province. 
RST receipts for the 2008/09 fiscal year totalled 
approximately $17.3 billion, net of $192 million in 
refunds ($16.2 billion in 2006/07), which repre- 
sents about 28% of the province’s total tax revenue. 


e Nostandardized province-wide criteria 
had been developed for selecting vendors 
for audit on the basis of the risk of non- 
compliance, despite the Ministry’s previous 
commitments to do so. 

e While audit coverage had increased since 
our last audit in 2000, the Ministry’s 

; coverage of each of its three categories of 

Over the last decade, RST revenues have increased 

vendors based on level of sales and amount 
by an average of about 5% to 6% annually. 


of tax remitted was still below its targets. 
In our 2007 Annual Report, we concluded that 


e Outstanding accounts receivable had increased 
to $967 million as of December 31, 2006, from 
$587 million at the time of our last audit in 


the enhanced information that ongoing technol- 
ogy developments can provide, along with certain 
improvements in the audit and collection processes, 1999/2000, an increase of approximately 654 
would all be necessary before the Ministry could be j 


In contrast, in the same period, RST revenues 
assured that all the RST owing was being collected. P 


had increased to approximately $16.2 billion 
from $12.6 billion, an increase of 29%. | 


@ The Ministry’s information system did not have 


Some of our more significant observations were: 
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@ While the Ministry had implemented certain 





measures to identify non-registered vendors 


; the ability to prioritize accounts-receivable for 
at their place of business or at points of sale, 


collection. Our review of a sample of open col- 
procedures were not yet adequate to ensure 


; ; lection files found that it often took a number 
that all Ontario vendors—particularly new Sik 

: : of months for a collector to initiate contact 
vendors—selling taxable goods and services 


. : wa on a file, and approximately one-quarter of 
were registered with the Ministry. i : “ 


the files had no coliéction activity for periods 
exceeding two years. 


430 


e At the time of our audit, approximately 
35,000 vendors with active accounts had been 
in default in filing their returns. Of those we 
reviewed, over eight months had elapsed, on 
average, between the referral of the account 
to the Ministry’s Non-Filer Unit and the com- 
pliance officer’s attempt to contact the vendor. 
After the initial contact, many files continued 
to have an extended period of inactivity. 

We made a number of recommendations for 
improvement and received commitments from the 
Ministry that it would take action to address our 
concerns. 


Status of Recommendations 





According to information received from the Min- 
istry of Revenue, significant progress has been 
made towards implementing many of the recom- 
mendations we made in our 2007 Annual Report. 
However, in 2009, the government announced that 
a single, federally administered sales tax—the Har- 
monized Sales Tax—was to be implemented in July 
2010. As noted throughout this follow-up, transi- 
tion plans need to ensure that improvements made 
in addressing the recommendations we made in our 
2007 Annual Report are not lost in the transition to 
the federal government. 

The status of action taken on each of our recom- 
mendations is as follows. 


OVERVIEW OF PROGRAM 
Tax Roll Maintenance 


Recommendation 1 
To help ensure that the tax roll for vendors that sell 
taxable goods and services is complete and accurate 
and that the appropriate amount of tax is remitted, 
the Ministry of Revenue should: 
@ ensure that it can match the government’s bust- 
ness names registry with its new management 
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information system—which would allow it to 
follow up with businesses that are on the names 
registry but not the RST vendor database; and 

@ at future meetings relating to interprovincial 
taxation, raise the possibility of reciprocal tax 
collection agreements with other provinces 
whereby all provincial sales taxes are collected 
at the point of sale and remitted to the province 
where the taxable goods are ultimately shipped 
and consumed. 


Status 
The Ministry has made some progress in the area 
of tax roll maintenance. The Ministry informed us 
that it had signed a Memorandum of Understanding 
with the Companies and Personal Property Secur- 
ity Branch of the Ministry of Government Services 
in April 2008, agreeing to receive semi-annually 
information relating to businesses under the Busi- 
ness Names Act and the Limited Partnership Act. The 
Ministry plans to review and match this information 
to that in its database so it can determine which 
vendors are not registered for RST purposes. The 
most recent information received covered the period 
ending October 2008. We understand that, subse- 
quent to our follow-up, the Ministry matched that 
information to its own database and found a high 
degree of correlation. The Ministry also relies on 
audit referrals, matching of federal data, and refer- 
rals from the Ministry of Labour for this purpose. 

In addition, the Ministry established an 

Interprovincial Best Practices Committee in 2008 
to discuss common areas of interest, including 
interprovincial reciprocal tax collection. The 
Ministry has also conducted a review of all existing 


agreements with other provinces to determine their 


effectiveness and any potential enhancements. In 
2009, the government announced that Ontario 
would be participating in the federally adminis- 
tered Harmonized Sales Tax. After this transition 
takes place, reciprocal tax collection is to be negoti- 
ated with the Canada Revenue Agency. 
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ENFORCEMENT: RST AUDITS 
Auditable Tax Roll 


Recommendation 2 
In order to ensure that potentially high-risk vendors 
are not systematically excluded from audit selection, 
we encourage the Ministry of Revenue to revise its 
audit selection process to include both newly regis- 
tered and part-time vendors. 


Status 
The Ministry informed us that it had acted on this 
recommendation by creating an automated audit- 
selection system called Flexible and Integrated Risk 
Systems (FAIRS), which was implemented in March 
2009. The Ministry uses the system to ensure that 
no vendors are excluded from the auditable tax roll 
for audit-selection purposes. 

The Ministry also demonstrated to us that it has 

included new registrants and part-time vendors in 
the audit-selection process. 


Audit Selection 


Recommendation 3 
To help ensure that it meets its goals of ensuring that 
the correct amount of tax is remitted to the province 
and of encouraging voluntary compliance in the 
broader vendor community, the Ministry of Revenue 
should: 

@ complete the development of an automated, cen- 
tralized evidence-based risk-assessment system 
for selecting vendors for audit and implement it 
as soon as possible; and 

@ specify the approximate number or percentage 
of higher-risk refunds issued subject to audit 
that are to be audited each year and ensure that 
the audits are carried out. 


Status 
As mentioned above, the Ministry implemented 
the automated risk-assessment and audit-selection 
system (FAIRS) in March 2009. This system is to 
include an enhancement, expected to be rolled out 


in October 2009, for the selection of refunds subject 
to audit. 

The Ministry has not specified what percentage 
of higher-risk refunds issued subject to audit is to be 
audited each year, as we recommended. However, 
the Ministry advised us that, in light of the RST pro- 
gram’s move to the federally administered Harmon- 
ized Sales Tax, all refunds issued by the Ministry 
subject to audit will be considered for review. In 
that regard, it has recently performed a manual risk 
assessment on all refunds issued subject to audit to 
determine their potential tax exposure and to assist 
in ensuring that audits are selected appropriately. 


Audit Coverage 


Recommendation 4 
To ensure that all vendors are given due consideration 
for audit selection and to encourage voluntary com- 
pliance through an adequate and representative level 
of audit coverage, the Ministry of Revenue should: 
@ continuously monitor its audit coverage for 
all three vendor categories and endeavour to 
meet its audit coverage goals for each as soon as 
possible; 
e select audits from all segments of the vendor 
population; and 
e facilitate the audit selection and results assess- 
ment process by reducing and more clearly 
defining the number of different vendor profile 
codes it uses. 


Status 

To support the wind-down of the RST program, 
the Ministry developed a strategy for audit selec- 
tion. This strategy focused on high-risk areas. In 
particular, the main focus was on large vendors, 
which account for 75% of the total tax remitted, 
and on those small and medium vendors that are 
considered to be high risk. 

To facilitate audit selection, the Ministry used 
the new system, FAIRS, to generate information on 
all of the large vendors (10,000 in total), review and 
assess the risk of each one, and select and prioritize 


audits accordingly. The Ministry also used FAIRS to 
identify small and medium vendors that are high 
risk; audits will be selected from those identified. 

The Ministry has not determined when it will 
conclude its audits of the RST program because the 
Canada Revenue Agency has not yet committed to 
performing audits after the transition. 

We were informed that the Ministry had com- 
pleted the transition from profile codes to North 
American Industry Classification System (NAICS) 
codes, which are consistent with the codes used 
by the Canada Revenue Agency. Although we 
recommended that the number of codes used be 
reduced, the new system has resulted in an increase 
from 255 to 790 codes. The Ministry told us this 
approach was necessary to classify businesses more 
specifically and was also a requirement of their new 
tax administration and audit software, ONT-TAXS. 
It has grouped similar codes into peer codes to 
facilitate better audit selection and risk evaluation. 


~ Audit Working Papers 


Recommendation 5 
To help ensure that the confidentiality of taxpayer 
information is maintained and provide evidence that 
audits have been adequately planned and conducted, 
the Ministry of Revenue should ensure that: 
@ all audit working paper files are securely stored 
and available for review; and 
@ audit working paper files contain the docu- 
mentation necessary to demonstrate that all 
required work has been adequately planned and 
completed, and reviewed and approved by an 
audit manager. 


Status 
The Ministry informed us that it has made the 
following improvements with regard to our recom- 
mendations on audit working papers: 
e In one location, the Ministry piloted an 

electronic working-paper process for all 

“no change” audits. The pilot was deemed 

a success and the process was subsequently 
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implemented at all locations. The Ministry 
then began to look for other types of audits 
for which working papers could be stored 
electronically. 

e The Ministry migrated to new tax admin- 
istration and audit software, ONT-TAXS, 
which allows all supporting documents to be 
scanned and attached to the file electronically. 
This permits complete electronic storage of 
the audit working papers. 

e It strengthened the policy on audit documen- 
tation, which is contained in the risk-based 
Integrated Audit Handbook released in 
June 2009. It also developed training, which 
is to be delivered in October 2009, to help 
managers and auditors increase their aware- 
ness of the documentation requirements of an 
audit file. 

@ It established in April 2009 an Audit Review 
unit, comprised of an audit manager and two 
senior operations officers, which is respon- 
sible for reviewing a sample of files after 
assessments have been issued for compliance 
with documentation standards and reporting 
to the Ministry’s senior management. The 
unit plans to begin these reviews in the third 
quarter of 2009/10. 

At the time of our follow-up, the Ministry had 
not yet addressed our concerns regarding the 
security of hardcopy audit working papers nor had 
it improved its processes around the review and 
approval of this information by an audit manager. 
However, subsequent to our follow-up, the Ministry 
informed us that it intends to add a module to its 
electronic files to record the audit managers’ review 
and approval of the working papers. 


Penalties 


Recommendation 6 
In order to deter taxpayers from remitting an incorrect 
amount of tax, the Ministry of Revenue should comply 
with its policy that penalties be imposed in all cases 
where an assessment was issued due to the taxpayer’s 
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neglect, carelessness, wilful default, or fraud, unless 
the reasons for not doing so are clearly documented 
and approved by the audit manager. 

In addition, in cases involving the potential 
imposition of a significant penalty (that is, exceeding 
a predetermined threshold amount), the Ministry 
should assess the merits of having more senior staff 
review the case and decide whether or not to impose 
the penalty. 


Status 
The Ministry indicated that it has fully imple- 
mented our recommendations on penalties. In 
June 2009, it released the risk-based Integrated 
Audit Handbook, which includes policies on pen- 
alty application and proper documentation and 
approval. An Audit Handbook Advisory Commit- 
tee, comprised of managers and auditors from all 
program areas, was also established and has been 
meeting regularly to discuss these policies. 

The Audit Review unit that the Ministry estab- 
lished in March 2009 in response to the above 
recommendation is to include a review of penalties 
as part of their documentation-standards review to 
ensure their consistent application and compliance 
with legislation and will report results to the Min- 
istry’s senior management. 


Monitoring of Audit Staff 


Recommendation 7 
In order to maximize productive audit hours and 
resultant audit assessments, the Ministry should: 

@ investigate the reasons for relatively high 
absenteeism rates among auditors and take the 
necessary corrective action; 

@ ensure that auditors comply with the Ministry’s 


flextime policy and limit time-banking deficits to 


no more than 14% hours at any point in time; 
@ continue to monitor auditors’ time charged 
to travel, with a view to further reducing time 
charged to travel; and 
e identify best practices and other strategies used 
by those auditors who consistently have high 
audit recovery rates. 


Status 
The Ministry informed us that it had made the fol- 
lowing improvements to the monitoring of audit 
staff since our 2007 Annual Report: 

e It committed to ensuring that all staff who 
exceed the Ministry’s absence threshold enter 
the Attendance Support Program and has 
been monitoring this process. 

e Since January 2008, it has been sending 
monthly reports on absenteeism to senior 
managers to follow up on. 

@ It has required that managers track absentee- 
ism through the government’s information 
system (WIN) and review timesheets on a 
monthly basis to ensure that the 14.5-hour 
time-banking limit has not been exceeded. 

e It has updated the ministry policy on time- 
reporting, including flextime and its limits. 
This policy is reiterated in the Integrated 
Audit Handbook to encourage compliance. 

e In March 2009, it established an Audit 
Handbook Advisory Committee, comprised 
of managers and auditors from all program 
areas, which meets regularly to identify and 
share best practices in audit methodologies to 
maximize revenue recovery. 

We reviewed an attendance report on short- 
term sickness absences provided by the Ministry 
for January to March of 2009 and noted that 
it included what actions were taken to address 
absences. The Ministry informed us that, although 
managers have been reviewing and monitoring 
travel time to identify staff with above-average time 
charged to travel, no significant reduction in the 
amount of hours charged to travel has been noted. 


COLLECTION FUNCTION 
Outstanding Accounts Receivable 


Recommendation 8 
To address the increase in outstanding accounts recetv- 
able, the Ministry of Revenue should be more proactive 


in taking prompt and rigorous collection action 


and ensure that all collection activity is adequately 
documented. 


Status 
The Ministry has made some progress in implement- 
ing our recommendations on collection efforts. 
Its new tax administration and audit software 
(ONT-TAXS) includes a collections function, which 
the Ministry began to use in November 2008. As a 
result, the Ministry now prioritizes its tax collectors’ 
workloads on the basis of a risk-scoring methodol- 
ogy. The Ministry informed us that it expected the 
new methodology to move the right account to the 
right collector, facilitating prompt and consistent col- 
lection action, and that its collection staff have been 
acting on 85% of accounts within a 90-day period. 
However, the outstanding accounts receivable bal- 
ance had increased to $1.05 billion as of March mel hs 
2009, from $967 million at the time of our audit, and 
a further $273 million had been written off. 

With regard to documentation, ONT-TAXS 
includes a function for documenting each stage of 
the collection process. This should help ensure that 
all activities performed by the collector are consist- 
ently and effectively noted in the system, including 
the results achieved. 


Overdue Returns and Non-filers 


Recommendation 9 
To give it the best chance of receiving outstanding RST 
returns and the required remittances, the Ministry of 
Revenue should ensure that: 
@ initial contact with defaulting vendors is made 
ona more timely basis; and 
@ after initial contact, follow-up with defaulting 
vendors is made on a continuous and timely 
basis until the matter is resolved. 


Status 
In January 2009, the Ministry began issuing auto- 
mated estimated tax assessments to Retail Sales 
Tax vendors that had not filed their RST returns on 
time. Any vendor in default is immediately issued 
an automated estimated tax assessment, and this 
process continues on a monthly basis. 
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The Ministry informed us that, from January 
to June 2009, it had issued approximately 83,000 
estimated assessments to approximately 63,000 
vendors, which resulted in $34 million in RST 
recoveries. The Ministry expects that the number 
of new RST referrals to the Collections branch as 
a result of returns in default will decrease over 
time as vendors become more educated about the 
requirement to file on time. 


SPECIAL INVESTIGATIONS 


Recommendation 10 
In order to ensure that all cases that warrant investi- 
gation are in fact investigated, and that the results of 
the investigations and any prosecutions are considered 
during future audits, the Ministry of Revenue should: 
@ obtain the level of staff required to ensure that 
all referrals that warrant investigation are in 
fact investigated; and 
@ analyze and, where warranted, communicate 
the results of investigations and prosecutions to 
all auditors and audit managers for considera- 
tion in their work. 


Status 
The Ministry informed us that its Special Investiga- 
tions branch had hired two additional staff mem- 
bers earlier this year to assist with investigations 
for the RST program. However, at the time of this 
follow-up, these resources were being redirected to 
higher priority or more pressing areas. As the RST 
program began winding down in anticipation of the 
new Harmonized Sales Tax, the investigators were 
to complete the RST investigations they have been 
assigned and only in extreme circumstances were 
they to take on any additional RST cases. 

The Ministry has also implemented a procedure 
to communicate the results of investigations and 
prosecutions to all auditors and audit managers. A 
memo with the results of each court case is issued 
within two weeks of receiving the notification from 
the Ministry’s prosecutors at the Legal Services 
Branch. 
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Background 





Ontario has 19 publicly funded universities (18 in 
2006), with full- and part-time enrolment in fall 
2008 totalling 448,000 (436,000 in 2006) and 
ranging from 1,000 to 74,000 students (3,400 to 
72,000 in 2006) per institution. In the year ended 
April 30, 2008, their operating revenues totalled 
about $6.3 billion, comprising $3.1 billion in prov- 
incial grants, $2.4 billion in tuition fees, and the 
balance from donations, investments, and miscel- 
laneous sources. Total operating expenditures were 
about $5.8 billion. 

Ontario universities own most of their facili- 
ties. A report published by the Council of Ontario 
Universities in 2007 stated that universities in this 
province managed a portfolio of 918 buildings 
with 5.6 million square metres of space, excluding 
student residences. The estimated replacement 
value of these facilities was $14.4 billion as of 
March 2007, while the value of associated infra- 
structure, such as boilers and power systems, was 
an estimated $2.2 billion. The average age of the 
buildings was over 30 years as of March 2007. 

As owners of their facilities, universities are 
responsible for utility costs and day-to-day clean- 
ing, repairs, and security services. The Ministry of 
Training, Colleges and Universities expects these 
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costs to be funded out of the universities’ operat- 
ing revenues. In addition to daily operating costs, 
universities are also responsible for maintaining 
the facilities in good condition. The Ministry assists 
universities with these costs through its Facilities 
Renewal Program grants of $26.7 million per year. 

Recognizing the increasing backlog of capital 
projects required to maintain university facilities in 
good condition and the need to have good informa- 
tion for decision-making, universities purchased a 
common capital-asset-management system in 2001. 
The system indicated that the backlog of deferred 
maintenance was estimated to be $1.6 billion as of 
March 2007. 

The objective of our 2007 audit was to assess 
whether universities had adequate processes in 
place to manage and maintain their academic 
and administrative facilities cost effectively. We 
examined the facility- management practices at 
three universities—Carleton University, McMaster 
University, and the University of Guelph. The other 
15 universities and the Ontario College of Art and 
Design completed a questionnaire about their poli- 
cies and practices. 

We found that the three universities would 
benefit from having better information about 
space utilization and about their physical-plant 
operations. 


At the three universities, we also found the 
following: 

e Inthe 2005/06 fiscal year, the combined cap- 
ital renewal projects at the three universities 
totalled $18.3 million—less than 5% of their 
combined deferred-maintenance amount, 
which was not sufficient to reduce the backlog 
of deferred maintenance. 

e The usefulness of the universities’ capital- 
asset-management system for prioritizing 
capital renewal projects could be enhanced by 
implementing procedures to update the sys- 
tem for completed renewal projects in a more 
timely manner and, for a sample of facilities, 
checking the reliability of the deferred main- 
tenance forecasts made by the system. 

e Procedures to ensure that academic and 
administrative space was used efficiently 
needed to be improved. A new scheduling 

. System at one university was expected to 
achieve a 30% improvement in the utilization 
of academic space. 

@ There was a need for additional analysis to 
compare the operating costs of each facility to 
those of similar facilities at the university or to 
those at other universities in order to identify 
and take action on opportunities to reduce 
costs. 

@ With respect to purchasing, we were pleased 
to note that the universities’ policies promoted 
open and competitive purchasing practices, 
and that the policies were generally being 
complied with for the purchases relating 
to the physical-plant operations that we 
examined. 

We made a number of recommendations for 
improvement and received commitments from the 
Ministry and the three universities that they would 
take action to address our concerns. 
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Status of Recommendations 





The three universities we visited and the Ministry 
provided us with information, as of spring 2009, 
on the status of the implementation of the recom- 
mendations in our 2007 Annual Report. The infor- 
mation provided indicated that, especially given 
the amount of work involved to fully address our 
recommendations, good progress was being made. 
The status of actions taken on each of our recom- 
mendations is as follows. 


RENEWAL OF FACILITIES 
Deferred Maintenance 


Recommendation 1 
To help ensure that decisions dealing with the main- 
tenance of university facilities are based on adequate 
information, universities should: 

@ periodically verify that the renewal models 
used by their capital-asset-management system 
are generating reliable deferred-maintenance 
forecasts; 

@ establish programs to periodically re-inspect the 
condition of their facilities; 

@ institute periodic, independent reviews to verify 
that their procedures meet the intent of the 
Facilities Condition Assessment Program; and 

@ maintain facility-condition information in their 
capital-asset-management database at a level 
of detail that is consistent with the way in which 
renewal projects are undertaken, and update 
the database as projects are completed. 

To help ensure that university facilities provide 
effective work and learning environments, the Min- 
istry of Training, Colleges and Universities should 
work with universities to develop a plan to reduce the 
extent of deferred maintenance. 


Status 
The Ministry stated that in June 2008 it had asked 
universities (and colleges) to submit campus facility 
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information to assist the Ministry in making capital 
funding decisions. The information requested 
included space utilization ratios, the value of 
deferred maintenance on and current replacement 
cost of their buildings, and capital projects prior- 
ities. The Ministry also advised us that, with the 
support of the Ministry of Energy and Infrastruc- 
ture, it was developing a comprehensive long-term 
capital plan and project evaluation methodology 

to address the ongoing capital investment require- 
ments, including deferred maintenance needs, of 
Ontario’s post-secondary institutions. This initiative 
includes engaging consultants to work with the 
management at post-secondary institutions to iden- 
tify and prioritize capital investment requirements 
and share best practices. The Ministry expected to 
complete this initiative in 2009. 

The Ministry had also provided universities with 
significant funding in addition to the $26.7 million 
annual facility renewal program grant—a one-time 
grant of $335 million in 2008 to reduce the deferred 
maintenance backlog and $427 million for 2009/10 
and 2010/11 in connection with the Knowledge 
Infrastructure Program, which can be used for both 
new buildings and renovation of old buildings. 

The three universities we visited had each taken 
action to implement some of the recommendations 
as follows: 

e As discussed in our 2007 report, Ontario’s 
universities jointly purchased the same 
capital-asset-management system, which pro- 
vides them with estimates of the cost of their 
deferred maintenance backlogs and forecasts 
of the timing of required capital renewal 
expenditures. One of the universities we 
visited advised us that it reviewed the system’s 
estimates of the cost and timing of various 
types of maintenance needs and found them 
to be accurate. The other two universities had 
begun to implement procedures to periodic- 
ally check system accuracy. 

e@ We were advised that all Ontario universities 
have agreed that building-condition databases 
on the capital-asset-management system will 


be updated, at a minimum, on a seven-year 
cycle. One of the universities we visited is 
updating its building-condition database on a 
five-year cycle, at a rate of 20% per year. The 
other two have reported that they are moving 
to the same system (one was completely re- 
auditing its entire building portfolio over the 
next two years, before moving to the five-year 
cycle). 

At the time of our follow-up, the three univer- 
sities we visited were in the process of input- 
ting building-condition data at a level of detail 
sufficient to enable them to update the data- 
base as each renewal project is completed, as 
opposed to waiting until the overall condition 
of the buildings concerned is periodically 
reassessed. In addition, Ontario universities 
have established an infrastructure committee 
to work with the vendor of the capital-asset- 
management system to enhance the system so 
that condition data on campus infrastructure 
such as water mains, sewers, sidewalks, roads, 
and street lighting can be input and used to 
generate estimates of deferred maintenance 
backlogs and to forecast required capital 
renewal expenditures for these assets as well. 
We were also advised that the universities 
have recently completed a request-for- 
proposal process to select a single facility- 
assessment firm to provide a consistent and 
cost-effective facility-audit and data-entry 
service for the Ontario University System. 
Over the next 12 months, approximately 25 
million square feet of space are to be aud- 
ited. The proposal included pricing for the 
standard system-level audit and a modified 
comprehensive audit that will respond to the 
recommendation that each institution collect 
data that is consistent with the way in which 
renewal projects are generally undertaken. 

If effectively implemented, it will also help 
ensure that procedures meet the intent of the 
Facilities Condition Assessment Program. Six 
of the participating institutions, totalling 14 


million square feet, have committed to the 
modified comprehensive audit. 


Prioritization of Renewal Projects 


Recommendation 2 
To help better ensure that capital-renewal funds are 
allocated to the highest-priority projects, universities 
should take steps to ensure that they have accurate 
and complete schedules of renewal projects due in 
each year and, where there are insufficient funds to 
complete all projects that are due, implement formal 
project-ranking procedures. 


Status 
As mentioned earlier, the universities we visited 
were in the process of inputting data to their build- 
ing-condition databases at a level of detail sufficient 
to enable them to update their databases as each 
renewal project is completed. They informed us 
that, as progress is made, this project would enable 
them to use one of their capital-asset-management 
system’s tools that is designed to support formal 
project-ranking processes. In addition, one of the 
universities plans to test a system modification that 
is intended to enable building-condition assessors 
to input risk factors as they conduct their assess- 
ments and thereby make the project-ranking tool 
more effective. 


UTILIZATION OF FACILITIES 


Recommendation 3 
To help ensure that they minimize their space needs 
and the associated facility costs, universities should: 
@ ensure that they have adequate systems and 
procedures to measure, analyze, and report on 
hours of use versus available hours, and space 
needed versus space used; and 
@ set space utilization objectives to be achieved 
over a three- to five-year time frame. 
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Status 
The three universities we visited have made varying 
degrees of progress in implementing these recom- 
mendations, as follows: 

@ One of the universities already had a manage- 
ment position responsible for space manage- 
ment at the time of our audit in 2007. This 
manager is responsible for maintaining its 
database inventory of classroom and labora- 
tory space and continuing to co-ordinate 
annual space reviews and space-requirement 
studies. The university was continuing to work 
toward space-utilization objectives previously 
recommended by consultants it had engaged 
to review this area. 

@ Another university hired a manager of space 
and capital planning and a space-planning 
technician to maintain the inventory of class- 
room and laboratory space and implement 
a space-management software system. The 
system is designed to enable the university 
to prepare a space-management report that 
compares the space used by each faculty to 
the space standards published by the Council 
of Ontario Universities. Starting in the fall of 
2009, space audits that will include analyses 
such as hours of use versus available hours 
are to be performed for each faculty. A Space 
Planning Committee has also been established 
to use this information as the basis for setting 
space-management policies that encompass 
utilization, allocation, and management 
issues. 

@ The other university has included responsibil- 
ity for utilization of classroom space in the 
duties of a new senior management position. 
This position will be responsible for oversee- 
ing a project to re-inventory classroom space 
and develop a space-utilization plan. 
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INFORMATION FOR CONTROLLING 
COSTS 


Recommendation 4 
To help manage facility costs, universities should 
implement systems and procedures to provide man- 
agement with the information required to: 

e enable them to take facility costs into account 
when making decisions, including those 
regarding the design and approval of new edu- 
cational programs and research projects; and 

@ perform both the internal- and external-cost 
comparisons required to identify poor and good 
practices, and take action to correct or promote 
them respectively. 


Status 
All three universities were participating in broad- 
based surveys of building costs and comparing 
their total operating costs by category to the 
published averages. Such comparisons are useful in 
identifying instances of above-average costs for the 
university as a whole in certain categories and may 
provide a basis for taking corrective action. 

We were advised that the Canadian Association 
of University Business Officers has initiated a joint 
project with the Association of Higher Education 
Facilities Officers (known as APPA) to establish a 
Canadian version of the APPA Facilities Perform- 
ance Indicators benchmarking system. The system 
is set up to collect facilities-operations data related 
to administration, construction, energy, mainten- 
ance, custodial and grounds services, the condition 
of facilities, and customer satisfaction. The Can- 
adian version is intended to support comprehensive 
facilities-operations benchmarking across Canada, 
and Ontario universities have endorsed participa- 
tion in the initiative. The universities were also 
participating in an energy benchmarking initiative 
with the Ontario Power Authority, the results of 
which were expected in fall 2009. 

The universities we visited had different levels 
of information about the operating costs of individ- 
ual buildings. Two universities had custodial and 
maintenance costs by building; one did not have a 


rigorous system for allocating these costs. Two had 
utility costs by building through extensive use of 
sub-metering; one had such information only for 
newer and renovated buildings. Such information is 
used to identify opportunities for savings by analyz- 
ing and comparing costs. For example, one univer- 
sity reported that analyzing power consumption has 
allowed them to plan energy conservation projects 
in a more strategic and focused way. 

Implementing our recommendations requires 
detailed analyses of the operating cost and utiliza- 
tion data of individual buildings to determine the 
impact on operating costs of various factors—such 
as hours of use, intensity of use, building design, 
and the type of finishing materials used in construc- 
tion—and thereby identify poor and good practices. 
The universities we visited did not have plans to 
perform such analyses. While we recognize that it 
may be practical to collect the necessary informa- 
tion only for newer and renovated buildings, some 
of the findings that result from analyzing data from 


_ these buildings may also help to control operating 


costs in older buildings. 


MONITORING PERFORMANCE AND 
QUALITY CONTROL 


Recommendation 5 
To help ensure that they receive value for the money 
they spend and that work is properly completed, unt- 
versities should: 

e consider establishing service-level objectives and 
require that their physical-plant and security 
departments report on the achievement of these 
objectives; 

@ implement supervisory inspections of the 
work of staff and contractors for quality and 
completeness, and document the results of these 
inspections; and 

@ use survey results and complaint informa- 
tion to help evaluate departmental and staff 
performance. 


Status 
The physical-plant departments of the universities 
we visited had expanded or initiated the measure- 
ment of the service levels they provided with 
respect to building custodial and maintenance ser- 
vices, and in one case groundskeeping. They meas- 
ured themselves against the five levels of service 
that the Association of Higher Education Facilities 
Officers has defined for each category. 

Although the initiatives of the physical-plant 
departments are useful, implementing our recom- 
mendation requires that the available information 
on service levels and related costs be used by 
universities to set service-level objectives that best 
balance a safe and productive working and learning 
environment against available funding. One uni- 
versity indicated that, although it is not yet setting 
service-level objectives for its physical-plant depart- 
ment, the most recent budget decisions ensured 
that the minimum custodial-service levels that 
the university believed to be necessary to provide 
acceptable learning and working environments 
were maintained. This is a step toward relating 
expenditures to particular service levels. 
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The three universities we visited also advised us 
that they were developing or expanding procedures 
to: 

@ perform and document inspections to verify 
that expected or contracted levels of service 
are being achieved—for example, one of the 
universities was equipping its custodial ser- 
vice managers with hand-held equipment to 
monitor and record performance; and 

@ use survey information to help evaluate 
departmental performance—at one of the 
universities, the faculty and students at one 
of the faculties designed and administered 
an extensive survey of satisfaction with the 
upkeep and cleanliness of all of the univer- 
sity’s facilities. 

One university indicated that it had also trained 
maintenance staff to use its management infor- 
mation system to more effectively manage work 
orders. It stated that its backlog of work orders 
older than 90 days had dropped by 75% as a result. 
It also implemented a more efficient way of trans- 
porting trades staff around campus that it believed 
would result in substantial cost savings. 
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INTRODUCTION 


In reviewing my Office’s activities this past year 

in regard to the Government Advertising Act, 2004 
(Act), I recalled the words spoken in December 
2004 by the then Chair of Management Board, 
who said, when the Bill was being passed into 

law: “Every dollar spent on partisan advertising 

is a dollar wasted—a dollar that doesn’t get spent 
on providing textbooks for students or reducing 
waiting times in emergency wards.” He also stated: 
“The bottom line is that any advertisement deemed 
by the Auditor General’s Office to promote parti- 
san interests will never see the light of day.” This 
chapter, which satisfies the legislative requirements 
in the Act as well as in the Auditor General Act to 
report annually to the Legislative Assembly, out- 
lines the work we have done over the past year to 
ensure that the principles enunciated by the Chair 
five years ago are adhered to. 


HISTORY 


The Act was introduced as Bill 25 at the end of 
2003, passed into law the following year, and took 
effect in December 2005—but was years in the 
making. As early as the mid-1990s, legislators had 
expressed concern about the appropriateness of a 
government’s use of public funds for advertising 
that could be considered to further partisan inter- 
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ests. In our 1999 Annual Report, we raised questions 
about government advertising of a “party-political 
nature” and about the appropriateness of the 
then-government’s use of public funds for certain 
advertising and communications campaigns. 

The report noted that Ontario had no “criteria 

to help distinguish between informative govern- 
ment advertising and party-political advertising.” 
We suggested that it would be “in the interest of 
improving public accountability for the government 
and/or the Legislature as a whole to consider the 
establishment of principles, guidelines, and criteria 
that clearly define the nature and characteristics of 
taxpayer-funded advertising.” 

Between 1999 and 2003, four private members’ 
bills were introduced, each seeking to provide a 
legislative framework for government advertising. 
In late 2004, the Legislative Assembly enacted the 
Government Advertising Act, 2004 (Act). Its main 
intent is to prohibit government advertising that 
may be viewed as promoting the governing party’s 
political interests by fostering a positive impression 
of the government or a negative impression of any 
group or person critical of the government. Under 
the Act, which can be found at www.e-laws. gov. 
on.ca, most government advertisements must be 
submitted to and approved by the Auditor General 
before they can be used. 


Overview of the Advertising 


Review Function 





Under the Act, the Auditor General is responsible 
for reviewing specified types of government 
advertising to ensure that they meet legislated 
standards and that, above all, they do not contain 
anything that is, or may be interpreted as being, 
primarily partisan in nature. The Act outlines vari- 
ous standards each advertisement must meet and 
states that “an item is partisan if, in the opinion 

of the Auditor General, a primary objective of the 
item is to promote the partisan political interests 
of the governing party.” The Act also provides the 
Auditor General with the discretionary authority to 
consider additional factors in determining whether 
a primary objective of an item is to promote the 
partisan political interests of the governing party 
(see “Other Factors”). 


WHAT FALLS UNDER THE ACT 


The Act currently applies to advertisements author- 
ized by government offices—specifically, govern- 
ment ministries, Cabinet Office, and the Office of 
the Premier. These offices must submit proposed 
advertising that is subject to the Act to my Office for 
review and approval before it can be used. 

The Act applies to advertisements that govern- 
ment offices will be paying to have published in a 
newspaper or magazine, displayed on a billboard, 
or broadcast on radio or television; and to printed 
matter that a government office proposes to pay to 
have distributed to households in Ontario either by 
bulk mail or by another method of bulk delivery. 
Advertisements meeting any of these definitions— 
no matter the language—are known as “review- 
able” items. 

The Act specifically excludes from review any 
advertisement or printed matter that is a job adver- 
tisement or a notice to the public required by law. 
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Also excluded are advertisements concerning the 
provision of goods and services to a government 
office and those regarding an urgent matter affect- 
ing public health or safety. 

Although the following are not specifically 
excluded by the Act, we have come to a mutual 
understanding with the government that they are 
not subject to the Act: 

e electronic advertising on government websites 
or any public site, except for web pages identi- 
fied and promoted in a reviewable item (see 
the Websites subsection later in this chapter); 
and 

e brochures, pamphlets, newsletters, news 
releases, consultation documents, reports, 
and other similar printed matter, materials, or 
publications. 


SUBMISSION AND USE OF ADVERTISING 
ITEMS 


Sections 2, 3, 4, and 8 of the Act require that gov- 
ernment offices submit every reviewable item to the 
Auditor General’s Office for review approval. The 
government office cannot publish, display, broad- 
cast, distribute, or disseminate the submitted item 
until the head of that office (that is, the deputy min- 
ister) receives notice, or is deemed to have received 
notice, that the advertisement has been approved. 

The Auditor General’s Office, by regulation, has 
seven business days within which to render its deci- 
sion. If we do not give notice within this time frame, 
the government office is deemed to have received 
notice that the item meets the standards of the Act, 
and it may run the advertisement. 

If the head receives notice from my Office that 
the item does not meet the Act’s standards, the item 
may not be used. However, the government office 
may submit a revised version of the rejected item 
for a further review. As with the first submission, 
my Office has seven days to render its decision. 





Once an item has been approved, a government 
office may use it for the next 12 months. Under the 
Act, all decisions of the Auditor General are final. 


STANDARDS FOR PROPOSED 
ADVERTISEMENTS 


In conducting its review, the Auditor General’s 
Office first determines whether the proposed adver- 
tisement—a reviewable item—meets the standards 
of the Act, as follows: 

@ The item must be a reasonable means of 
achieving one or more of the following 
objectives: 

e toinform the public of current or proposed 
government policies, programs, or services; 

e to inform the public of its rights and 
responsibilities under the law; 

e to encourage or discourage specific social 
behaviour in the public interest; and/or 

e to promote Ontario, or any part of the prov- 
ince, as a good place to live, work, invest, 
study, or visit, or to promote any economic 
activity or sector of Ontario’s economy. 

e The item must include a statement that it is 
paid for by the government of Ontario. 

e The item must not include the name, voice, or 
image of a member of the Executive Council 
(cabinet) or a member of the Legislative 
Assembly (unless the primary target audience 
is located outside Ontario, in which case the 
item is exempt from this requirement). 

@ The item must not have as a primary objective 
the fostering of a positive impression of the 
governing party, or a negative impression of a 
person or entity critical of the government. 

e The item must not be partisan; that is, in the 
opinion of the Auditor General, it cannot have 
as a primary objective the promotion of the 
partisan political interests of the governing 


party. 
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OTHER FACTORS 


In addition to the specific statutory standards 
above, the Act allows the Auditor General to 
consider additional factors to determine whether 
a primary objective of an item is to promote the 
partisan political interests of the governing party 
[subsection 6(4)]. In general, the additional factors 
that we consider relate to the general impression 
conveyed by the message and how it is likely to 
be received or perceived. In determining whether 
an item may be perceived or received as partisan, 
consideration is given to whether it includes certain 
desirable characteristics and avoids certain undesir- 
able ones, as follows: 

e@ Each item should: 

e contain subject matter relevant to govern- 
ment responsibilities (that is, the govern- 
ment should have direct and substantial 
responsibilities for the specific matters 
dealt with in the item); 

e present information objectively, in tone and 
content, with facts expressed clearly and 
accurately, using unbiased and objective 
language; 

e emphasize facts and/or explanations, not 
the political merits of proposals; and 

e enable the audience to distinguish between 
fact on the one hand and comment, opin- 
ion, or analysis on the other. 

e Items should not: 

e use colours, logos, and/or slogans com- 
monly associated with any recognized 
political party in the Legislative Assembly 
of Ontario; 

e inappropriately personalize (for instance, 
by attacking opponents or critics); 

e directly or indirectly attack, ridicule, or 
criticize the views, policies, or actions of 
those critical of the government; 

e be aimed primarily at rebutting the argu- 
ments of others; 

e intentionally promote, or be perceived as 
promoting, political-party interests (to 


this end, consideration is also given to 
such matters as timing of the message, the 
audience it is aimed at, and the overall 
environment in which the message will be 
communicated); 

e deliver self-congratulatory or political- 
party image-building messages; 

e deal with matters such as a policy proposal 
where no decision has yet been made, 
unless the item provides a balanced 
explanation of both the benefits and the 
disadvantages; 

e present pre-existing policies, products, ser- 
vices, or activities as if they were new; or 

e use a uniform resource locator (URL) to 
direct readers, viewers, or listeners to a 
“first click” web page with content that may 
not meet the standards required by the Act 
(see Websites). 


OTHER REVIEW PROTOCOLS 


Since taking on responsibility for reviewing govern- 
ment advertising, my Office has endeavoured to 
clarify, in co-operation with government offices, 
areas where the Act is silent. What follows is a brief 
discussion of the main areas that have required 
clarification over the years. 


Websites 


Although websites are not specifically designated 
as reviewable under the Act, we believe that a 
website mentioned in an advertisement should be 
seen as an extension of the ad. Following discus- 
sions about this with the government, we came 

to an agreement that the first page or “click” of a 
website accessed by using the URL featured in a 
reviewable item would be included in our review. 
We agreed not to consider web pages beyond the 
first click. However, we review the first-click page 
for any information or messages that may not meet 
the standards of the Act. For example, the first-click 
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web page must not include a minister’s name, voice, 
or photograph, nor deliver self-congratulatory, 
party image-building messages, or messages that 
attack the policies, opinions, or actions of others. 


Event/Conference Program Advertisements 
and Payments in Kind 


Government advertisements sometimes appear 

in programs and other materials distributed at 
public events such as conferences, trade shows and 
exhibitions. In considering this type of advertise- 
ment, we concluded that it should be subject to the 
Act because the programs usually follow the same 
format and serve a similar purpose as magazines 
and other print media. In other words, advertise- 
ments are interspersed with content even though 
such ad space is at times provided to a government 
office free of charge. On the issue of payment for 
the advertisement, government offices often make 
in-kind or financial contributions to an event, 
including paid sponsorship. Therefore, we consider 
the “free” advertisement to have been indirectly 
paid for. 

Our rationale was based on the fact that the 
free advertisement would typically not have been 
granted if the government office had not made a 
financial contribution or sponsored the event. Gov- 
ernment officials have agreed with our approach to 
advertisements in programs distributed at public 
events. Consequently, items in these programs must 
be submitted for review. 


Third-party Advertising 


Government funds provided to third parties are 
sometimes used for the purpose of advertising. The 
government and my Office have agreed that, for 
third-party advertising, the advertising must be 
submitted for review if it meets all of the three fol- 
lowing criteria: 

@ a government office provides the third party 

with funds intended to pay part or all of the 


cost of publishing, displaying, broadcasting, 
or distributing the item; 

e the government grants the third party permis- 
sion to use the Ontario logo or another official 
provincial visual identifier in the item; and 

e the government office approves the content of 
the item. 


Government Recruitment Advertisements 


As previously noted, the Act specifically excludes 
job advertisements from review. We have inter- 
preted this exemption to apply to advertising for 
specific government jobs, but not to broad-ranging 
generic recruitment campaigns, such as ads for the 
recruitment of medical professionals in Ontario. 
The government has agreed with our interpreta- 
tion. As a result, generic recruitment campaigns 
must be submitted to my Office for review. 


Environmental Assessment Notices 


The Act exempts from review any government 
notices required by law. Nevertheless, the Ministry 
of Natural Resources used to routinely submit for 
review and approval advertisements for certain 
classes of environmental assessment notices for 
provincial parks and conservation reserves. We 
discussed this with ministry representatives and 
came to an agreement that, because of the statutory 
nature of these advertisements, they do not require 
clearance through my Office. 


Pre-reviews and Consultations 


A pre-review is available to government offices 
wishing to have us examine an early version of an 
item. This can be a script or storyboard, provided 
that it reasonably and accurately reflects the 

item as it is intended to appear when completed. 
Pre-reviews help limit the investment of time and 
money spent to develop items containing material 
that could be deemed objectionable under the Act. 
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If material submitted for pre-review appears to 
violate the Act, we provide explanatory comments 
to the government office. If it appears to meet the 
standards of the Act, we so advise the government 
office. However, before the item can be published, 
displayed, broadcast, printed, or otherwise dis- 
seminated, the government office must submit the 
finished item for review to ensure that it still meets 
the standards of the Act. 

A pre-review is strictly voluntary on our part and 
is outside the statutory requirements of the Act. 


External Advisers 





Under the Auditor General Act, the Auditor Gen- 
eral can appoint an Advertising Commissioner to 
assist in fulfilling the requirements of the Govern- 
ment Advertising Act, 2004. However, instead of 
appointing an Advertising Commissioner, my Office 
has engaged external advisers to provide assistance 
and advice in the ongoing review of items submit- 
ted for review. The following advisers have been 
engaged at various times by my Office during the 
2008/09 fiscal year: 

e Rafe Engle is a Toronto lawyer who specializes 
in advertising, marketing, communications, 
and entertainment law. He is also the outside 
legal counsel for Advertising Standards Can- 
ada. Before studying law, Mr. Engle acquired 
a comprehensive background in media and 
communications while working in the adver- 
tising industry. 

e Jonathan Rose is Associate Professor of 
Political Studies at Queen’s University. He is 
a leading Canadian academic with interests 
in political advertising and Canadian politics. 
Professor Rose has written a book on govern- 
ment advertising in Canada and a number of 
articles on the way in which political parties 
and governments use advertising. 


@ Joel Ruimy is a Toronto communications 
consultant with many years of experience as 
a journalist, editor, and producer covering 
Ontario politics in print and television. 
These advisers provided invaluable assistance 
in our review of government advertising this past 
fiscal year. 


Advertising Review Activity, 


2008/09 





RESULTS OF OUR REVIEWS 


During the 2008/09 fiscal year, we reviewed 889 
individual advertising items, with a total value of 
more than $52 million. We provided our decision in 
all cases within the required seven-day period. The 
length of time required for a review and decision 
can vary, depending on the complexity of the mes- 
sage and on the other work priorities of our review 
panel. Nevertheless, our average turnaround time 
during the past fiscal year was 3.5 business days. 

We also received and reviewed 14 pre-review 
submissions that were at a preliminary stage of 
development, most often at the script or storyboard 
level. Because pre-reviews are strictly voluntary on 
our part and outside the statutory requirements 
of the Act, they are second in priority to finished 
items. Nonetheless, we make every effort to com- 
plete the pre-reviews within a reasonable length of 
time. The average turnaround time for pre-review 
submissions in the 2008/09 fiscal year was about 
four business days. 

Of all the submissions reviewed, we rejected 
five submissions, comprising 15 advertisements, 
because they did not meet the Act’s standards, 
some of which fostered a positive impression of 
the governing party. All but one of these advertise- 
ments were subsequently revised, resubmitted, and 
approved. 
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Although we did not note any contraventions of 
the Act—advertisements that ran without having 
been submitted to us for review—we noted a few 
instances of third-party advertising that seemed 
problematic. Such advertisements fell into two 
main categories: ads issued by a third party that 
receives funding and/or direction from a govern- 
ment office and that communicated a strong 
Ontario government brand in that they made prom- 
inent use of the Ontario logo or mentioned the 
support or sponsorship of the Ontario government; 
and ads issued by a third party receiving taxpayer 
funds that communicated a congratulatory message 
to the government. 

All advertisements in the first category would 
have been granted approval had our ad panel 
reviewed them. However, we were still concerned 
that ads communicating a strong government of 
Ontario brand will be mistaken as government of 
Ontario ads, and there is a risk that a similar type 
of ad that does not meet the standards could run in 
the future. Therefore, we have initiated discussions 
with the government offices involved and hope 
such ads will in future be submitted for review. 

As for the second category, we were concerned 
about one transit advertisement, given its con- 
gratulatory message to the government for a recent 
infrastructure investment together with the prom- 
inent use of the Ontario logo. During our inquiries 
into this issue, we were told that the third party 
that initiated the ad provided staff in the Premier’s 
office with advance copies of the ad. Our inquiries 
also revealed that there was no concern expressed 
about the congratulatory partisan message being 
delivered by the third party because the ad was not 
subject to the Act. Although we agreed that this 
ad falls outside the scope of the Act, we were still 
concerned that no attempt was made to discourage 
the third party from the partisan message in the 
ad. We recommend that the government consider 
instituting a prohibition on congratulatory partisan 
advertising by third parties that receive Ontario 
government funding directly or indirectly. 
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We were also concerned that use of the Ontario 
logo in this ad could leave the impression that the 
ad was somehow sanctioned or endorsed by the 
government. We were told that neither the fund- 
ing ministry nor the agency responsible for the 
government’s visual identity program were aware 
of the ad in question. When we first brought the 
ad to their attention, we would have expected the 
government to have expressed concern to the third 
party about the use of the Ontario logo, given that 
government policy requires that the use of the logo 
be authorized by the government. Since then, the 
- government has indicated that, in future, it will 
endeavour to more rigourously enforce its policy on 
the use of the Ontario logo. 


EXPENDITURES ON ADVERTISEMENTS 
AND PRINTED MATTER 


The Auditor General Act requires that the Auditor 
General report annually to the Legislative Assembly 
on expenditures for advertisements, printed mat- 
ter, and messages that are reviewable under the 
Government Advertising Act, 2004. 

Figure 1 contains expenditure details of indi- 
vidual advertising campaigns reported to us by 
each ministry for media-buy costs; agency creative 
costs; third-party production, talent, and distribu- 
tion costs; and other third-party costs, such as 
translation. 

In order to test the completeness and accuracy 
of the reported advertising expenditures, my Office 
reviewed randomly selected payments to suppliers 
of advertising and creative services and their sup- 
porting documentation at selected ministries. We 
also performed certain compliance procedures with 
respect to the requirements of sections 2, 3, 4, and 8 
of the Government Advertising Act, 2004, which 
pertain to submission requirements and prohibition 
on the use of items pending the Auditor General’s 
review. We found no matters of concern in our 
review work. 


Other Matter 





PROPOSED AMENDMENTS TO THE 
GOVERNMENT ADVERTISING ACT, 2004 


On March 26, 2009, the government introduced 
Bill 162, the Budget Measures Act, 2009, which 
contained, among other things, proposed amend- 
ments to the Government Advertising Act, 2004 
(Act). Some of the proposed amendments would 
have marginally widened the scope of advertising 
items covered by the Act to include cinema ads; 
others, however, would have rewritten section 6 of 
the Act to eliminate some of the existing standards 
for government advertising and to revoke the 
Auditor General’s discretionary powers to consider 
additional factors in determining whether an ad is 
partisan. The proposed amendments also included 
a more narrow and limited definition of what could 
be considered partisan. 

I acted immediately to express my Office’s 
concerns about the amendments to the Minister of 
Finance. The most significant of my concerns was 
that my Office would no longer be able to exercise 
discretion or use professional judgment on the 
issue of partisanship. The government responded 
quickly, and a number of meetings and discussions 
were held between my staff and senior govern- 
ment officials. These meetings established that 
our concerns and the proposed amendments were 
irreconcilable in the short term. As a result, the 
government decided to postpone amendments until 
such time as further consultation and discussion 
could take place. On May 14, 2009, the government 
withdrew the proposed amendments during debate 
of Bill 162 in the Standing Committee on Finance 
and Economic Affairs. 

Over the course of the discussions that took 
place about the amendments, my Office maintained 
that the Act is effective in its current form and that 
it contributes to ensuring that taxpayer dollars are 


not used to fund partisan government advertis- 

ing. In fact, my Office has hosted several visiting 
legislators from Australia who wished to discuss the 
workings of the Act because there was great interest 
in their jurisdictions to possibly enact similar legis- 
lation. According to our research and conversations 
with legislators in other jurisdictions, the Act in its 
current form is regarded as the international gold 
standard for government-advertising legislation. 
That said, we would welcome the opportunity to 
continue discussions on possible future amend- 
ments to the Act to make it even more effective and 
transparent. 
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Figure 1: Expenditures for Reviewable Advertisements and Printed Matter under the 
Government Advertising Act, 2004, April 1, 2008-March 31, 2009 


Source of data: Ontario government offices 
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Third-party Costs ($) cont’d Media Costs ($) Ad Value** Campaign 
E)) Total ($) 
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* Out-of-Home advertising includes, for example, billboards and transit posters. 
** Ad Value denotes the value of an ad space provided to government offices at no cost, 
often where the government has provided funding for a related event/publication. 
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# of # of Agency | Third-party Costs ($) 


Submissions Items Costs ($) 
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Third-party Costs ($) cont’d Media Costs ($) Ad Value** Campaign 
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* Out-of-Home advertising includes, for example, billboards and transit posters. 


** Ad Value denotes the value of an ad space provided to government offices at no cost, 
often where the government has provided funding for a related event/ publication. 
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* Out-of-Home advertising includes, for example, billboards and transit posters. 
** Ad Value denotes the value of an ad space provided to government offices at no cost, 
often where the government has provided funding for a related event/ publication. 
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ad developed by another ministry, but used here 
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* Out-of-Home advertising includes, for example, billboards and transit posters. 
** Ad Value denotes the value of an ad space provided to government offices at no cost, 
often where the government has provided funding for a related event/publication. 
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Chapter 6 


Appointment and 


Composition of the 
Committee 





The Standing Orders of the Legislative Assembly 
provide for the appointment of an all-party Stand- 
ing Committee on Public Accounts. The Committee 
is appointed for the duration of the Parliament 
(that is, the period from the opening of the first ses- 
sion immediately following a general election to the 
end of a government’s term and the next general 
election). 

The membership of the Committee reflects 
proportionately the representation of parties in the 
Legislative Assembly. All members except the Chair 
are entitled to vote on motions; the Chair may vote 
only to break a tie. 

In accordance with the Standing Orders, a 
Standing Committee on Public Accounts was 
appointed on December 10, 2007, for the duration 
of the 39th Parliament. The membership of the 
Committee when the House adjourned for the sum- 
mer recess on June 4, 2009, was as follows: 

Norm Sterling, Chair, Progressive Conservative 

Jerry Ouellette, Vice-chair, Progressive 

Conservative 

Laura Albanese, Liberal 

France Gélinas, New Democrat 

Ernie Hardeman, Progressive Conservative 

Phil McNeely, Liberal 


The Standing Committee 
on Public Accounts 


Liz Sandals, Liberal 
Maria Van Bommel, Liberal 
David Zimmer, Liberal 


Role of the Committee 


The Committee examines, assesses, and reports 





to the Legislative Assembly on a number of issues, 
including the economy and efficiency of govern- 
ment operations; the effectiveness of programs 

in achieving their objectives; controls over assets, 
expenditures, and the assessment and collection of 
revenues; and the reliability and appropriateness of 
information in the Public Accounts. 

In fulfilling this role, pursuant to its terms of ref- 
erence in the Standing Orders of the Assembly, the 
Committee reviews the Auditor General’s Annual 
Report and the Public Accounts, holds a number of 
hearings throughout the year, and reports to the 
Legislative Assembly its observations, opinions, 
and recommendations. Under the Standing Orders, 
the Auditor General’s Annual Reports and the 
Public Accounts are deemed to have been perma- 
nently referred to the Committee as they become 
available. 

In addition, under sections 16 and 17 of the 
Auditor General Act, the Committee may request the 
Auditor General to undertake a special assignment 
in an area of interest to the Committee. 


AUDITOR GENERAL'S ADVISORY ROLE 
WITH THE COMMITTEE 


In accordance with section 16 of the Auditor Gen- 
eral Act, the Auditor General and senior staff attend 
committee meetings to assist the Committee in its 
review and hearings related to the Auditor Gen- 
eral’s Annual Report and the Public Accounts. 


Committee Procedures and 





Operations 


GENERAL 


The Committee meets weekly when the Legislative 
Assembly is sitting. With the approval of the House, 
it may also meet at any time when the Legislative 
Assembly is not sitting. All meetings are open to the 
public with the exception of those dealing with the 
setting of the Committee’s agenda and the prepara- 
tion of committee reports. All public committee 
proceedings are recorded in Hansard (the official 
verbatim report of debates in the House, speeches, 
other proceedings in the Legislative Assembly, 

and all open-session sittings of standing and select 
committees). 

The Committee selects matters from the Audi- 
tor General’s Annual Report for hearings. These 
matters typically relate to the Auditor General’s 
value-for-money audit work. The Auditor General, 
along with the Committee’s researcher, briefs the 
Committee on these matters, and the Committee 
then requests senior officials from the auditee to 
appear and respond to questions at the hearings. 
Since the Auditor General’s Annual Report deals 
with operational, administrative, and financial 
rather than policy matters, ministers rarely attend. 
Once the hearings are completed, the Committee 
reports its comments and recommendations to the 
Legislative Assembly. 

The Committee also follows up on when and 
how the ministries, Crown agencies, and organiza- 
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tions in the broader public sector not selected for 
hearings will address the concerns raised in the 
Auditor General’s Annual Report. This process 
enables each auditee to update the Committee on 
what it has done in response to the Auditor Gen- 
eral’s recommendations since the completion of the 
audit. 


MEETINGS HELD 


The Committee met 22 times during the October 
2008—June 2009 period to review the special report 
on Prevention and Control of Hospital-acquired Infec- 
tions issued in September 2008 and the following 
sections from the Auditor General’s 2008 Annual 
Report, and to write reports, where warranted, for 
subsequent tabling in the Legislative Assembly: 

e Adult Institutional Services; 

e Brampton Civic Hospital Public—private Part- 
nership Project; 
Child and Youth Mental Health Agencies; 
Community Mental Health; 
Employment and Training Division; 
Gasoline, Diesel-fuel, and Tobacco Tax; 
Ontario Clean Water Agency; 
Special Education; and 


School Boards—Acquisition of Goods and 
Services (follow-up of 2006 audit). 


REPORTS OF THE COMMITTEE 


The Committee issues its reports to the Legisla- 
tive Assembly. These reports summarize the 
information reviewed by the Committee during 
its meetings, as well as make comments and 
recommendations. 

All committee reports are available through 
the Clerk of the Committee (or online at www. 
ontla.on.ca), thus providing the public with full 
access to the findings and recommendations of the 
Committee. 

After the Committee tables a report in the 
Legislative Assembly, it requests that ministries or 
agencies respond to each recommendation either 
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within 120 days or within a time frame stipulated 
by the Committee. 

During the period from September 2008 to 
July 2009, the Committee submitted the following 
reports to the Legislative Assembly: 

e Archives of Ontario and Information Storage 

and Retrieval Services; 

e Fish and Wildlife Program; 

e Hazardous Waste Management; 

e Hospitals—Management and Use of Surgical 

Facilities; 

e@ Ontario Sex Offender Registry; 

e@ Outbreak Preparedness and Management; 

@ Prevention and Control of Hospital-acquired 

Infections; and 
e@ Universities—Management of Facilities. 


FOLLOW-UP ON RECOMMENDATIONS 
MADE BY THE COMMITTEE 


The Clerk of the Committee is responsible for fol- 
lowing up on the actions taken by ministries, agen- 
cies, and organizations in the broader public sector 
in response to the Committee’s recommendations. 
The Office of the Auditor General reviews responses 
from ministries and agencies and, in subsequent 
audits, follows up on the actions reported. 


COMMITTEE MOTION TO CONSIDER 
ISSUING SPECIAL REPORT 


On June 3, 2009, the Committee passed the follow- 
ing motion: 


That, following the Auditor General’s 
completion of his value-for-money audit 
of eHealth Ontario, the Standing Commit- 
tee on Public Accounts of the Legislative 
Assembly of Ontario calls on the Auditor 
General to release that chapter of his 
Annual Report in a special report to the 
Speaker; and that, prior to the tabling 

of this report, the Auditor General may 
inform the Deputy Ministry of Health and 
Long-Term Care of his opinions, observa- 
tions, or recommendations. 


Because of this motion by the Committee, as 
well as a similar request by the Minister of Health 
and Long-Term Care on June 2, 2009, the Audi- 
tor General submitted a special report entitled 
Ontario’s Electronic Health Records Initiative to the 
Speaker of the House in early fall 2009. 


OTHER COMMITTEE ACTIVITIES 


Canadian Council of Public Accounts 
Committees 


The Canadian Council of Public Accounts Commit- 
tees (CCPAC) consists of delegates from federal, 
provincial, and territorial public accounts commit- 
tees from across Canada. CCPAC meets at the same 
time and place as the Canadian Council of Legisla- 
tive Auditors (CCOLA) so that issues of mutual 
interest can be discussed. The 30th annual meeting 
of CCPAC was hosted by Alberta and was held in 
Edmonton from September 13 to 15, 2009. 





The Office of the Auditor General of Ontario 
(Office) serves the Legislative Assembly and the 
citizens of Ontario by conducting value-for-money 
and financial audits and reviews and reporting on 
them. By doing this, the Office helps the Legislative 
Assembly hold the government, its administrators, 
and grant recipients accountable for how prudently 
they spend public funds and for the value they 
obtain, on behalf of Ontario taxpayers, for the 
money spent. 

The work of the Office is performed under the 
authority of the Auditor General Act. In addition, 
under the Government Advertising Act, 2004, the 
Auditor General is responsible for reviewing and 
deciding whether or not to approve certain types of 
proposed government advertising (see Chapter 5 
for more details on the Office’s advertising review 
function). Both acts can be found at www.e-laws. 
gov.on.ca. 


General Overview 


VALUE-FOR-MONEY AUDITS IN THE 
ANNUAL REPORT 


About two-thirds of the Office’s work relates to 
value-for-money auditing. The Office’s value-for- 
money audits are assessments of how well a given 
“auditee” (the entity that we audit) manages and 
administers its programs or activities. The auditees 
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that the Office has the authority to conduct value- 
for-money audits of are: 

@ Ontario government ministries; 

@ Crown agencies; 

@ Crown-controlled corporations; and 

@ organizations in the broader public sector 
that receive government grants (for example, 
agencies that provide mental-health services, 
children’s aid societies, community colleges, 
hospitals, long-term-care homes, school 
boards, and universities). 

The Auditor General Act (Act) [in subclauses 
12(2) (f Civ) and (v)] identifies the criteria to be 
considered in this assessment: 

@ Money should be spent with due regard for 

economy. 

@ Money should be spent with due regard for 
efficiency. 

e@ Appropriate procedures should be in place to 
measure and report on the effectiveness of 
programs. 

Note that we assess whether or not the auditee’s 
management is evaluating—using appropriate per- 
formance measures—the effectiveness of programs 
and reporting on its findings. It is not part of our 
mandate to do these things. Rather, our mandate 
dictates that we report instances where we have 
noted that the auditee has not satisfactorily done its 
job in this area. 

The Act requires that, if the Auditor General 
observes instances where the three value-for-money 
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criteria have not been met, he or she report on 
them. The Act also requires that he or she report on 
instances where the following was observed: 

e Accounts were not properly kept or public 
money was not fully accounted for. 

e Essential records were not maintained or the 
rules and procedures applied were not suf- 
ficient to: 

e safeguard and control public property; 

e check effectively the assessment, collec- 
tion, and proper allocation of revenue; or 

e ensure that expenditures were made only 
as authorized. 

e@ Money was expended other than for the pur- 
poses for which it was appropriated. 

Assessing the extent to which the auditee was 
controlling against these risks is technically “com- 
pliance” audit work but is generally incorporated 
into both value-for-money audits and “attest” audits 
(discussed in a later section). Other compliance 
work that is typically included in our value-for- 
money audits is: 

e identifying the key provisions in legislation 
and the authorities that govern the auditee or 
the auditee’s programs and activities as well 
as those that the auditee’s management is 
responsible for administering; and 

e performing the tests and procedures we deem 
necessary to obtain reasonable assurance 
that the auditee’s management has complied 
with these key legislation and authority 
requirements. 

Government programs and activities are the 
result of government policy decisions. Thus, we 
could say that our value-for-money audits focus on 
how well management is administering and execut- 
ing government policy decisions. It is important to 
note, however, that in doing so we do not comment 
on the merits of government policy. Rather, it is the 
Legislative Assembly that holds the government 
accountable for policy matters. The Legislative 
Assembly continually monitors and challenges gov- 
ernment policies through questions during legisla- 


tive sessions and through reviews of legislation and 
expenditure estimates. 

In planning, performing, and reporting on our 
value-for-money work, we follow the relevant 
professional standards established by the Canadian 
Institute of Chartered Accountants. These stan- 
dards require that we have processes for ensuring 
the quality, integrity, and value of our work. Some 
of the processes we use are described below. 


Selecting What to Audit 


The Office audits major ministry programs and 
activities at approximately five- to seven-year inter- 
vals. We do not audit organizations in the broader 
public sector and Crown-controlled corporations on 
the same cycle because there are such a great num- 
ber of them and their activities are so numerous 
and diverse. Since our mandate expanded in 2004 
to allow us to audit these auditees, our audits have 
covered a wide range of topics in several sectors, 
including health (hospitals, long-term-care homes, 
and mental-health service providers), education 
(school boards, universities, and colleges), and 
social services (Children’s aid societies and social 
service agencies), as well as several large Crown- 
controlled corporations. 

In selecting what program, activity, or organ- 
ization to audit each year, we consider how great 
the risk is that an auditee is not meeting the three 
value-for-money criteria and therefore incurring 
potential negative consequences for the public it 
serves. To help us choose higher-risk audits, we 
consider factors such as: 

e the results of previous audits and related 

follow-ups; 

e the total revenues or expenditures involved; 

e the impact of the program, activity, or organ- 

ization on the public; 

e the complexity and diversity of the auditee’s 

operations; 

e recent significant changes in the auditee’s 

operations; and 


e the significance of the issues an audit might 
identify. 

We also consider whether the benefits of con- 
ducting the audit justify the costs of the audit. 

Another factor we take into account in the selec- 
tion process is what work the auditee’s internal 
auditors have completed or planned. Depending on 
what that work consists of, we may defer an audit 
or change our audit’s scope to avoid duplication of 
effort. In other cases, we do not diminish the scope 
of our audit but rely on and present the results of 
internal audit work in our audit report. 


Setting Audit Objectives, Audit Criteria, 
and Assurance Levels 


When we begin an audit, we set an objective for 
what we want to achieve. We then develop suitable 
audit criteria that cover the key systems, poli- 

cies, and procedures that should be in place and 
operating effectively. Developing criteria involves 
extensively researching sources such as recognized 
bodies of experts; other bodies or jurisdictions 
delivering similar programs and services; manage- 
ment’s own policies and procedures; applicable 
criteria successfully applied in other audits or 

_ reviews; and applicable laws, regulations, and 
other authorities. 

To further ensure their suitability, the criteria 
we develop are discussed with the senior manage- 
ment responsible for the program or activity at the 
planning stage of the audit. 

The next step is designing and conducting tests 
and procedures to address our audit objective 
and criteria, so that we can reach a conclusion 
regarding our audit objective and make observa- 
tions and recommendations. Each audit report has 
a section entitled “Audit Objective and Scope,” in 
which the audit objective is stated. 

Conducting tests and procedures to gather 
information has its limitations. We therefore cannot 
provide what is called an “absolute level of assur- 
ance” that our audit work identifies all significant 
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matters. Other factors also contribute to this. For 
example, we may conclude that the auditee had a 
control system in place for a process or procedure 
that was working effectively to prevent a particular 
problem from occurring; but auditee management 
or staff are often able to circumvent such control 
systems—so we cannot guarantee that the prob- 
lem will never arise. Also, muth of the evidence 
available for concluding on our objective is more 
persuasive than it is conclusive, and we must rely 
on professional judgment in much of our work—for 
example, in interpreting information. 

For all these reasons, the assurance that we plan 
for our work to provide is at an “audit level”—the 
highest reasonable level of assurance that we can 
obtain using our regular audit procedures. Spe- 
cifically, an audit level of assurance is obtained by 
interviewing management and analyzing the infor- 
mation it provides; examining and testing systems, 
procedures, and transactions; confirming facts 
with independent sources; and, where necessary 
because we are examining a highly technical area, 
obtaining expert assistance and advice. 

With respect to the information that manage- 
ment provides, under the Act we are entitled to 
have access to all relevant information and records 
necessary to the performance of our duties. Out of 
respect for the principle of Cabinet privilege, we 
do not seek access to the deliberations of Cabinet. 
However, the Office can access virtually all other 
information contained in Cabinet submissions or 
decisions that we deem necessary to fulfill our 
responsibilities under the Act. 

Infrequently, the Office will perform a review 
rather than an audit. A review provides a moder- 
ate level of assurance, obtained primarily through 
inquiries and discussions with management; 
analyses of information management provides; and 
only limited examination and testing of systems, 
procedures, and transactions. We perform reviews 
when, for example, providing a higher level of 
assurance has prohibitive costs, the Auditor General 
Act does not allow for a certain program or activity 
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to be audited, or other factors relating to the nature 
of the program or activity make a review more 
appropriate than an audit. 


Communicating with Management 


To help ensure the factual accuracy of our observa- 
tions and conclusions, staff from our Office com- 
municate with the auditee’s senior management 
throughout the value-for-money audit or review. 
Before beginning the work, our staff meet with 
management to discuss the objective and criteria 
and the focus of our work in general terms. During 
the audit or review, our staff meet with manage- 
ment to review progress and ensure open lines 

of communication. At the conclusion of on-site 
work, management is briefed on the preliminary 
results of the work. A draft report is then prepared 
and discussed with the auditee’s senior manage- 
ment. The auditee’s management provides written 
responses to our recommendations, and these are 
discussed and incorporated into the draft report. 
The Auditor General finalizes the draft report (on 
which the Chapter 3 section of the Annual Report 
will be based) with the deputy minister or head of 
the agency, corporation, or grant-recipient organ- 
ization, after which the report is published in the 
Annual Report. 


SPECIAL REPORTS 


As required by the Act, the Office reports on its aud- 
its in an Annual Report to the Legislative Assembly. 
In addition, the Office may make a special report to 
the Legislative Assembly at any time, on any matter 
that, in the opinion of the Auditor General, should 
not be deferred until the Annual Report. 

Two sections of the Act authorize the Auditor 
General to undertake additional special work. 
Under section 16, the Standing Committee on Pub- 
lic Accounts may resolve that the Auditor General 
must examine and report on any matter respecting 
the Public Accounts. Under section 17, the Legisla- 


tive Assembly, the Standing Committee on Public 
Accounts, or a minister of the Crown may request 
that the Auditor General undertake a special assign- 
ment. However, these special assignments are not 
to take precedence over the Auditor General’s other 
duties, and the Auditor General can decline such 

an assignment requested by a minister if he or she 
believes it conflicts with other duties. 

In recent years when we have received a special 
request under section 16 or 17, our normal practice 
has been to obtain the requester’s agreement that 
the special report will be tabled in the Legislature 
on completion and made public at that time. 

Our audit of eHealth, which began in fall 2008, 
was originally planned for inclusion in this Annual 
Report and was part of a collaborative initiative 
involving several Canadian auditors general to 
examine spending and progress on eHealth initia- 
tives federally and in several provinces. However, 
because public concerns were raised about spend- 
ing and the use of consultants at the eHealth 
Ontario agency, the Minister of Health and Long- 
Term Care requested that we expedite our audit 
and report it separately under section 17 of the Act. 

Accordingly, the Auditor General reported the 
results of the audit of eHealth to the Minister of 
Health and Long-Term Care and to the Legislature 
in early fall 2009. 

On August 31, 2009, the Minister of Energy 
and Infrastructure requested the Auditor General 
to examine expenses incurred by employees of 
the Ontario Lottery and Gaming Corporation. The | 
results of this audit will similarly be reported to the 
Minister and to the Legislature on completion. 


ATTEST AUDITS 


Attest audits are examinations of an auditee’s 
financial statements. In such audits, the auditor 
expresses his or her opinion on whether the finan- 
cial statements present information on the auditee’s 
operations and financial position in a way that 

is fair and that complies with certain accounting 


policies (in most cases, with Canadian generally 
accepted accounting principles). As mentioned in 
the overview of value-for-money audits, compliance 
audit work is often incorporated into attest audit 
work. Specifically, we assess the controls for man- 
aging risks relating to improperly kept accounts; 
unaccounted-for public money; lack of recordkeep- 
ing; inadequate safeguarding of public property; 
deficient procedures for assessing, collecting, 

and properly allocating revenue; unauthorized 
expenditures; and not spending money on what it is 
intended for. 


The Auditees 


Every year, we audit the financial statements of the 
province and the accounts of many agencies of the 
Crown. Specifically, the Act [in subsections 9(1), 
(2), and (3)] requires that: 

e the Auditor General audit the accounts and 
records of the receipt and disbursement of 
public money forming part of the province’s 
Consolidated Revenue Fund, whether held in 
trust or otherwise; 

e the Auditor General audit the financial state- 
ments of those agencies of the Crown that are 
not audited by another auditor; 

e public accounting firms that are appointed 
auditors of certain agencies of the Crown 
perform their audits under the direction of the 
Auditor General and report their results to the 
Auditor General; and 

e public accounting firms auditing Crown- 
controlled corporations deliver to the Auditor 
General a copy of the audited financial state- 
ments of the corporation and a copy of the 
accounting firm’s report of its findings and 
recommendations to management (typically 
contained in a management letter). 

Chapter 2 discusses this year’s attest audit of the 
province’s consolidated financial statements. 

We do not discuss the results of attest audits of 
agency and Crown-controlled corporations in this 
report. Agency legislation normally stipulates that 
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the Auditor General’s reporting responsibilities are 
to the agency’s board and the minister(s) respon- 
sible for the agency. Our Office also provides copies 
of the audit opinions and of the related agency 
financial statements to the deputy minister of the 
associated ministry, as well as to the Secretary of 
the Treasury Board. 

Where an agency attest audit notes areas where 
management must make improvements, the auditor 
prepares a draft management letter and discusses 
it with senior management. The letter is revised 
to reflect the results of that discussion. After the 
draft management letter is cleared and the agency’s 
senior management responds to it in writing, the 
auditor prepares a final management letter, which 
is usually discussed with the agency’s audit com- 
mittee. If a matter were so significant that we felt 
it should be brought to the attention of the Legisla- 
ture, we would include it in an annual report. 

Exhibit 1, Part 1 lists the agencies that were 
audited during the 2008/09 audit year. The Office 
currently contracts with public accounting firms 
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to audit a number of these agencies on the Office’s 
behalf. Exhibit 1, Part 2, and Exhibit 2 list the 
agencies of the Crown and the Crown-controlled 
corporations, respectively, that public accounting 
firms audited during the 2008/09 audit year. 


OTHER STIPULATIONS OF THE 
AUDITOR GENERAL ACT 


The Auditor General Act came about with the pas- 
sage, on November 22, 2004, of Bill 18, the Audit 
Statute Law Amendment Act, which received Royal 
Assent on November 30, 2004. The purpose of Bill 
18 was to make certain amendments to the Audit 
Act to enhance the ability of the Office to serve the 
Legislative Assembly. The most significant amend- 
ment contained in Bill 18 was the expansion of the 
Office’s value-for-money audit mandate to organ- 
izations in the broader public sector that receive 
government grants. This 2009 Annual Report marks 
the fourth year of our expanded audit mandate. 
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Appointment of Auditor General 


Under the Act, the Auditor General is appointed as 
an officer of the Legislative Assembly by the Lieu- 
tenant Governor in Council—that is, the Lieutenant 
Governor appoints the Auditor General on and with 
the advice of the Executive Council (the Cabinet). 
The appointment is made “on the address of the 
Assembly,” meaning that the appointee must be 
approved by the Legislative Assembly. The Act also 
requires that the Chair of the Standing Committee 
on Public Accounts—who, under the Standing 
Orders of the Legislative Assembly, is a member 

of the official opposition—be consulted before the 
appointment is made (for more information on the 
Committee, see Chapter 6). 


Independence 


The Auditor General and staff of the Office are 
independent of the government and its administra- 
tion. This independence is an essential safeguard 
that enables the Office to fulfill its auditing and 
reporting responsibilities objectively and fairly. 

The Auditor General is appointed to a 10-year, 
non-renewable term, and can be dismissed only for 
cause by the Legislative Assembly. Consequently, 
the Auditor General maintains an arm’s-length dis- 
tance from the government and the political parties 
in the Legislative Assembly and is thus free to fulfill 
the Office’s legislated mandate without political 
pressure. 

The Board of Internal Economy—an all-party 
legislative committee that is independent of the 
government’s administrative process—reviews and 
approves the Office’s budget, which is subsequently 
laid before the Legislative Assembly. As required 
by the Act, the Office’s expenditures relating to the 
2008/09 fiscal year have been audited by a firm of 
chartered accountants, and the audited financial 
statements of the Office are submitted to the Board 
and subsequently must be tabled in the Legislative 
Assembly. The audited statements and related dis- 
cussion of expenditures for the year are presented 
at the end of this chapter. 


CONFIDENTIALITY OF WORKING PAPERS 


In the course of our reporting activities, we prepare 
draft audit reports and management letters that are 
considered to be an integral part of our audit work- 
ing papers. It should be noted that these working 
papers, according to section 19 of the Auditor Gen- 
eral Act, do not have to be laid before the Legislative 
Assembly or any of its committees. As well, because 
our Office is exempt from the Freedom of Informa- 
tion and Protection of Privacy Act, our draft reports 
and audit working papers, which include all infor- 
mation obtained during the course of an audit from 
the auditee, cannot be accessed from our Office, 
thus further ensuring confidentiality. 


CODE OF PROFESSIONAL CONDUCT 


The Office has a Code of Professional Conduct to 
encourage staff to maintain high professional stan- 
dards and ensure a professional work environment. 
The Code is intended to be a general statement of 
philosophy, principles, and rules regarding conduct 
for employees of the Office, who have a duty to 
conduct themselves in a professional manner and to 
strive to achieve the highest standards of behaviour, 
competence, and integrity in their work. 

The Code explains why these expectations exist 
and further describes the Office’s responsibilities to 
the Legislative Assembly, the public, and our aud- 
itees. The Code also provides guidance on disclo- 
sure requirements and the steps to be taken to avoid 
conflict-of-interest situations. All employees are 
required to complete an annual conflict-of-interest 
declaration. 


Office Organization and 





Personnel 


The Office is organized into portfolio teams—a 
framework that attempts to align related audit 
entities and to foster expertise in the various areas 


of audit activity. The portfolios, which are loosely 
based on the government’s own ministry organiza- 
tion, are each headed by a Director, who oversees 
and is responsible for the audits within the assigned 
portfolio. Assisting the Directors and rounding out 
the teams are a number of audit Managers and vari- 
ous other audit staff (see Figure 1). 

The Auditor General, the Deputy Auditor Gen- 
eral, the Directors, and the Manager of Human 
Resources make up the Office’s Senior Management 
Committee. 


Canadian Council of 


Legislative Auditors 





This year, Alberta hosted the 37th annual meeting 
of the Canadian Council of Legislative Auditors 
(CCOLA) in Edmonton, from September 13 to 15, 
2009. This annual gathering has, for a number of 
years, been held jointly with the annual conference 
of the Canadian Council of Public Accounts Com- 
mittees. It brings together legislative auditors and 
members of the Standing Committees on Public 
Accounts from the federal government and the prov- 
inces and territories, and provides a useful forum for 
sharing ideas and exchanging information. 


International Visitors 





As an acknowledged leader in value-for-money 
auditing, the Office periodically receives requests 
to meet with visitors and delegations from abroad 
to discuss the roles and responsibilities of the Office 
and to share our value-for-money and other audit 
experiences with them. During the audit year 
covered by this report, the Office met with legisla- 
tors/public servants/auditors from China, Ghana, 
Kenya, the Republic of Serbia, and the Russian 
Federation, as well as a delegation from the Com- 
monwealth nations. 
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Results Produced by the 


Office This Year 





The 2008/09 fiscal year was a challenging but suc- 
cessful year for the Office. 

In total, we conducted 14 value-for-money and 
special audits this year, together with a review of 
the status of the unfunded liability of the Work- 
place Safety and Insurance Board. Our value-for- 
money audits examined a wide range of services 
of importance to Ontarians. They included bridge 
safety, telehealth, eHealth, consumer protection, 
research funding, and efforts to measure and raise 
student literacy and numeracy. We also examined: 
programs that serve some of Ontario’s most vulner- 
able citizens, such as infection control in long-term- 
care homes, assistive devices, two major income 
support programs (the Ontario Disability Support 
Program and Ontario Works), and social housing. 
Also—for the first time—we looked at the whole 
issue of government user fees. 

Several of the value-for-money audits we carried 
out this year explored how the province oversees 
services that it partially pays for but that muni- 
cipalities provide. The delivery of Ontario Works 
assistance, the provision and maintenance of social 
housing, and the safety and maintenance of munici- 
pal bridges are examples. Our work in the broader 
public sector included examining the infection- 
control practices in three long-term-care homes, 
visiting several school boards to discuss efforts 
to improve student achievement by them and the 
Ministry of Education’s Literacy and Numeracy Sec- 
retariat, and auditing the administration of student 
testing conducted by the Education Quality and 
Accountability Office. We also spoke with several 
educational institutions about research funding and 
had discussions with both educational institutions 
and hospitals on the OntarioBuys program. 

As mentioned in the earlier Special Reports sec- 
tion, we issued a special report on Ontario’s Elec- 
tronic Health Records Initiative in early fall 2009. 
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Figure 1: Office Organization, September 30, 2009 


Annemarie Wiebe, Manager 
Shayna Whiteford 


Deputy Auditor General Professional Practices 


Michael Brennan, Manager 


Jim McCarter 





Gary Peall 





| Operations 


John Sciarra, Director 
Administration 

Shanta Persaud 

Maureen Bissonnette 

Sohani Myers 

Christine Wu 
Communications and Government Advertising Review 

Andréa Vanasse, Manager 

Shirley Falkner 

Mariana Green 

Tiina Randoja 
Information Technology 

Peter Lee 

Shams Ali 


Audit Porfolios and Staff 


Community and Social Services, Crown Agencies a 
and Revenue 


Walter Bordne, Director 
Wendy Cumbo, Manager 
Nick Stavropoulos, Manager 
Vishal Baloria 

Johan Boer 

Stephanie Chen 
Constantino De Sousa 
Katrina Exaltacion 


Health and Health Promotion 


Rudolph Chiu, Director 
Sandy Chan, Manager 

Denise Young, Manager 
Ariane Chan 

Frederick Chan 

Anita Cheung 

Suzanne Darmanin 

Lisa Li 


Paul Amodeo, Director 
Bill Pelow, Manager 
Tanmay Gupta 

Shariq Saeed 

Joyce Yip 


Inna Guelfand 
Aldora Harrison 
Li-Lian Koh 
Maria Zuyev 


Oscar Rodriguez 
Pasha Sidhu 
Alla Volodina 
Celia Yeung 
Gigi Yip 


Public Accounts, Finance, and — ; 
Information Technology _ 





Economic Development, Environment, Natural 
Resources, and Education and Training 











John McDowell, Director 
Walter Allan, Manager 
Tom Chatzidimos 


Gerard Fitzmaurice, Director 


Fraser Rogers, Manager 
Tony Tersigni, Manager 


Kandy Fletcher Tino Bove Mark Smith 

Mary Romano Maggie Dong Zhenya Stekovic 

Megan Sim Zahra Jaffer Ellen Tepelenas 
Mythili Kandasamy Dora Ulisse 
Joane Mui Brian Wanchuk 
Roger Munroe Jing Wang 


Health and Long-term-care Providers - 


Susan Klein, Director 
Laura Bell, Manager 
Vanna Gotsis, Manager 


~ Justice and Regulatory 


Vince Mazzone, Director 
Rick MacNeil, Manager 
Vivian Sin, Manager 





Naomi Herberg, Manager Helen Chow Cynthia Lau 
Emanuel Tsikritsis, Manager Howard Davy Sarah Noble 
Kevin Aro Justin Hansis Rashmeet Gill Alice Nowak 
Matthew Brikis Veronica Ho Mark Hancock Ruchir Patel 
Jennifer Fung Linde Qiu Alfred Kiang Janet Wan 
Ingrid Goh Gloria Tsang 


Transportation, Infrastructure, and oy 
Municipal Affairs 


Andrew Cheung, Director 
Teresa Carello, Manager 
Gus Chagani, Manager 


Kim Achoy Marcia DeSouza 
Bartosz Amerski Isabella Ho 

Izabela Beben Gajalini Ramachandran 
Kim Cho Alexander Truong 


Our decision to issue this audit report as a special 
report was prompted by a request by the Minister 
of Health and Long-term Care under Section 17 of 
our Act. 

As mentioned in the earlier Attest Audits sec- 
tion, we are responsible for auditing the province’s 
consolidated financial statements (further dis- 
cussed in Chapter 2), as well as the statements of 
more than 40 Crown agencies. We again met all of 
our key financial-statement audit deadlines while 
continuing our investment in training to success- 
fully implement ongoing revisions to accounting 
and assurance standards and methodology for con- 
ducting our financial-statement audits. A practice 
inspection by the Institute of Chartered Account- 
ants of Ontario confirmed that we were meeting the 
new standards in all significant respects. 

We successfully met our review responsibilities 
under the Government Advertising Act, 2004, as 
further discussed in Chapter 5. 

The results produced by the Office this year 
would clearly not have been possible without the 
hard work and dedication of our staff, as well as the 
assistance of our contract staff and expert advisors. 
With a number of senior staff retiring or on parental 
leave, contract staff were particularly important to 
us this year, and they filled in admirably. 


The Office of the Auditor General of Ontario \ 469 





Financial Accountability 


The following discussion and our financial state- 
ments outline the Office’s financial results for the 
2008/09 fiscal year. 

Figure 2 provides a comparison of our approved 
budget and expenditures over the last five years. 
Figure 3 presents the major components of our 
spending and shows that nearly 71% (also 71% 
in 2007/08) related to salary and benefit costs for 
our staff, while professional and other services and 
rent comprised most of the remainder. The propor- 
tions in Figure 3 have remained relatively constant 
in recent years, with the possible exception of 
contracted professional services. These services 
increased significantly again this year to help us 
manage the volume, timing, and complexity of our 
work and to temporarily replace retiring staff and 
new parents on leave. 

Overall, our expenses increased 3.8% (13.8% 





in 2007/08) and were again significantly under 
budget. Over the five-year period presented in 
Figure 2, we have returned unspent appropriations 
totalling almost $7.7 million. The main reason for 
this is that we have historically faced challenges in 
hiring and retaining qualified professional staff in 


Figure 2: Five-year Comparison of Spending (Accrual Basis) ($ 000) 


Prepared by the Office of the Auditor General of Ontario 


2004/05 





salaries and benefits 7,261 
professional and other services 877 
rent 891 
travel and communications 290 


other . bes 533 





a 14 







2005/06 2006/07 2007/08 = 2008/09 
-12,552-s«13,992s«15,308 = 16,245 
8,047 8,760 9,999 10,279 
951 1,264 1,525 1,776 
962 985 1,048 1,051 
324 363 397 332 
756 930 1,033 1,096 
AA 040 





‘47307. 45608 


*These amounts are typically slightly higher than the excess of revenue over expenses as a result of non-cash expenses 


(such as amortization of capital assets). 
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Figure 3: Spending by Major Expenditure Category, 
2007/08 


Prepared by the Office of the Auditor General of Ontario 


travel and 


communications (2.3%) 
other (7.5%) 






rent (7.2%) 


professional and 
other services 
(12.2%) 


salaries and 
benefits (70.7%) 


the competitive Toronto job market—our public- 
service salary ranges have simply not kept pace 
with compensation increases for such professionals 
in the private sector. A more detailed discussion of 
the changes in our expenses and some of the chal- 
lenges we are facing follows. 


SALARIES AND BENEFITS 


Our salary and benefit costs rose just 2.8% this 
year. Salary and performance pay increases (in line 
with those approved for Ontario public servants), 
together with benefit cost increases (such as higher 
pension and health benefit contribution rates), 
were partially offset by a decrease in the number 
of staff employed compared to last year anda 
decrease in our future benefit obligations (owing to 
the retirement of several senior staff and the payout 
of our previously expensed obligation in respect of 
these employees). 

Following a gradual increase in approved com- 
plement over the last several years—to 117 from 90 
(see Figure 4)—we were able to gradually increase 
the average number of staff we employ to 110. By 


early in the year our staffing actually peaked at 
115 but with turnover and retirements declined to 
just 106 by the end of 2008/09—about the same 
number we began the previous year with. With the 
economic uncertainty and need for cost contain- 
ment through much of this year, we were reluctant 
to staff up when staff departed and instead made 
more extensive use of contract professionals. As a 
result, our average staffing over the course of this 
year was about the same as last year. However, 

our hiring continues to be primarily at more junior 
levels, given that our salaries and benefits are 
competitive at these levels. We quickly fall behind 
private- and broader-public-sector salary scales for 
more experienced professional accountants. This 
is one reason that, as Figure 4 shows, we still have 
a number of unfilled positions. The growing com- 
plexity of our audits demands that we use highly 
qualified, experienced staff as much as possible. 
The challenge of maintaining and enhancing our 
capacity to perform these audits will only increase 
as more of our most experienced staff retire over 
the next few years. 

Under the Act, our salary levels must be compar- 
able to the salary ranges of similar positions in the 
government. These ranges remain uncompetitive 
with the salaries that both the not-for-profit and the 
private sectors offer. According to the most recent 
survey by the Canadian Institute of Chartered 
Accountants published in 2009, average salaries 
for CAs in government ($117,700) are 15% lower 





Figure 4: Staffing, 2004/05-2008/09 
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than those in the not-for-profit sector ($138,400) 
and, most importantly, 27% lower than those work- 
ing for professional service CA firms ($160,600), 
which are our primary competitors for professional 
accountants. This gap has narrowed only slightly 
since the previous survey in 2007. 

The salaries of our highest-paid staff in the 2008 
calendar year are disclosed in Note 6 to our finan- 
cial statements. 


RENT 


Our costs for accommodation were virtually the 
same as last year, increasing just 0.4% (owing 
primarily to rising building operating costs, particu- 
larly taxes and utilities). Accommodation costs con- 
tinue to decline as a percentage of total spending. 


PROFESSIONAL AND OTHER SERVICES 


These services represent our most significant cost 
pressure—they have increased $251,000, or more 
than 16%, from last year and have more than 
doubled since 2004/05. The largest component of 
the increase is the costs for, first, contract profes- 
sionals, and second, contract CA firms. 

We continue to have to rely more on contract 
professionals to meet our legislated responsibil- 
ities. It continues to be difficult for us to reach our 
approved full complement given our uncompeti- 
tive salary levels, more complex work, and tighter 
deadlines for finalizing the financial-statement 
audits of Crown agencies and the province. As men- 
tioned earlier, this year we had a number of staff 
on parental leave, as well as several retirements, 
which further increased our reliance on contract 
staff. We also believe that using more contract staff 
to fill temporary needs is a prudent approach to 
staffing, particularly during uncertain economic 
times, in that it provides more flexibility and less 
disruption if significant in-year cuts to our budget 
are requested. 

We continue to incur higher contract costs 
for CA firms we work with because of the higher 





The Office of the Auditor General of Ontario 471 


salaries they pay their staff and the additional 
hours required to implement ongoing changes to 
accounting and assurance standards. The full-year 
impact of the cost of contracting out two additional 
financial-statement audits last year also contributed 
to the increase in professional services costs. 


TRAVEL AND COMMUNICATIONS 


With less value-for-money audit work in broader- 
public-sector organizations, particularly hospitals, 
than last year, our travel costs actually declined 

by 16% this year. About half of our work last year 
focused on several different broader-public-sector 
service providers, including hospitals, children’s aid 
societies, mental-health agencies, and employment 
agencies. We also did a special audit of AgriCorp 

in Guelph. Consequently, our travel expenditures 
were quite high last year. Although we are incur- 
ring significantly more travel costs than in the past 
because of the expansion of our mandate to audit 
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broader-public-sector organizations, this year our 
audits focused more on ministry oversight of ser- 
vice providers and less on the providers themselves. 
This means that our teams made shorter visits to 
service providers. Staff also made greater use of 
technology to reduce travel requirements and costs. 


OTHER 


Other costs include asset amortization, supplies 
and equipment maintenance, training, and statu- 
tory expenses. Such costs increased by $63,000, 
or by 6%, over last year. Some of the increase 
($22,000) relates to higher equipment amortiza- 
tion owing to prior investments in computer and 
leasehold improvements, and a further $16,000 
relates to statutory salary and performance pay 
increases for the Auditor General that were in line 
with increases provided to senior deputy ministers 
in the government. About $13,000 of the increase 
relates to higher costs for software support licences 
and maintenance associated with our exchange 
server upgrade, including data encryption and 
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wireless security software licensing, as well as a net increase of about $9,000. Our training costs 
for toner cost and usage. We had greater need for increased by a modest 1.6% this year, or by $3,000, 
expert assistance to meet our responsibilities under _ but have risen by 55% over the last two years. These 
the Government Advertising Act, 2004, including the _ increased expenditures on training have helped to 
cost of independent research into how closely our ensure that our staff are able to adhere to the many 
judgments about partisanship in the advertising recent changes in standards and have increased 

we review mirror public opinion. This increase was their level of subject expertise to handle complex 
somewhat offset by lower statutory costs for other value-for-money audits. 

expert assistance on our VFM audits, resulting in 
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FINANCIAL STATEMENTS 


Office of the Auditor General of Ontario 
Bureau du vérificateur général de |’'Ontario 


tastes 
Nd 


MANAGEMENT’S RESPONSIBILITY FOR FINANCIAL STATEMENTS 


The accompanying financial statements of the Office of the Auditor General for the year 
ended March 31, 2009 are the responsibility of management of the Office. Management has 
prepared the financial statements to comply with the Auditor General Act and with Canadian 
generally accepted accounting principles. 


To ensure the integrity and objectivity of the financial data, management maintains a system 
of internal controls that provide reasonable assurance that transactions are appropriately 
authorized, assets are adequately safeguarded, appropriations are not exceeded and financial 
information is reliable and accurate. 
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The financial statements have been audited by the firm of Adams & Miles LLP, Chartered 
Accountants. Their report to the Board of Internal Economy, stating the scope of their 
examination and opinion on the financial statements, appears on the following page. 


we Ru 


Jim McCarter, FCA Gary R. Peall, CA 
Auditor General Deputy Auditor General 


Box 105, 15th Floor 
20 Dundas Street West 
Toronto, Ontario 

M5G 202 
416-327-2381 

fax 416-326-3812 


B.P. 105, 15° étage 

20, rue Dundas ouest 
Toronto (Ontario) 

M5G 2C2 

416-327-2381 
télécopieur 416-326-3812 


www.auditor.on.ca 
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501-2550 Victoria Park Ave 200-195 County Court Blvd 
AC ne 5} Toronto, ON M2J SA9 Brampton, ON L6W 4P7 
Ar ADAMS & MILES LLI Tel 416.502.2201 Tel 905-459.5605 


Chartered Accountants Fax 416 502.2210 Fax 905 459.2893 


AUDITOR’S REPORT 


To the Board of Internal Economy 
of The Legislative Assembly of Ontario 


We have audited the statement of financial position of the Office of the Auditor General 
of Ontario as at March 31, 2009 and the statements of operations and accumulated 
deficit and cash flows for the year then ended. These financial statements are the 
responsibility of the management of the Office of the Auditor General of Ontario. Our 
responsibility is to express an opinion on these financial statements based on our audit. 


We conducted our audit in accordance with Canadian generally accepted auditing 
standards. Those standards require that we plan and perform an audit to obtain 
reasonable assurance whether the financial statements are free of material 
misstatement. An audit includes examining, on a test basis, evidence supporting the 
amounts and disclosures in the financial statements. An audit also includes assessing 
the accounting principles used and significant estimates made by management, as well 
as evaluating the overall financial statement presentation. 





In our opinion, these financial statements present fairly, in all material respects, the 
financial position of the Office of the Auditor Generalof Ontario as at March 31,2009 
and the results of its operations and its cash flows for the year then ended in 
accordance with Canadian generally accepted accounting principles. 


The budget information is unaudited and not considered as part of the financial 
statements on which we have expressed our opinon. 


Adams & Miles LLP 


Chartered Accountants 
Licensed Public Accountants 


Toronto, Canada 
July 16, 2009 


An independent firm associated 
www.adamsmiles.com WithAGN International Ltd 








The Office of the Auditor General of Ontario 475 





Office of the Auditor General of Ontario 
Statement of Financial Position 














As at March 31, 2009 
2009 2008 
$ $ 
Assets 
Current 
Cash 293,306 521,868 
Due from Consolidated Revenue Fund 663,149 374,144 
956,455 896,012 
Capital Assets (Note 3) 581,060 598,271 
Total assets 1,537,515 1,494,283 
Liabilities 
Accounts payables and accrued liabilities 1,590,455 1,680,012 
Accrued employee benefits obligation [Note 4(B)] 1,997,000 2,014,000 ; is 
12 
Net assets (Accumulated deficit) s 
Investment in capital assets (Note 3) 581,060 598,271 eee 
Accumulated deficit [Note 2(B)] (2,631,000) (2,798,000) 


(2,049,940) (2,199,729) 
Total liabilities and accumulated deficit 1531,010 1,494,283 


Commitment (Note 5) 


See accompanying notes to financial statements. 


Approved by the Office of the Auditor General of Ontario: 


a a oe 


Jim McCarter Gary Peall 
Auditor General : Deputy Auditor General 
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Office of the Auditor General of Ontario 
Statement of Operations and Accumulated Deficit 


For the Year Ended March 31, 2009 





Revenue 


Consolidated Revenue Fund - Voted appropriation 


Expenses 
Salaries and wages 
Employee benefits (Note 4) 
Office rent 
Professional and other services 
Amortization of capital assets 
Travel and communication 
Training and development 
Supplies and equipment 
Transfer payment: CCAF-FCVI Inc. 
Statutory expenses: Auditor General Act 


Government Advertising Act 


Statutory services 
Total expenses (Note 7) 


Excess of revenue over expenses 
Less: returned to the Province 


Net deficiency/ (excess) of revenue over expenses (Note 2B) 


Accumulated deficit, beginning of year 
Accumulated deficit, end of year 


See accompanying notes to financial statements. 


2009 
Budget 
$ 


16,244,700 


9,588,200 
2,272,300 
1,053,400 
1,640,700 


418,800 
387,700 
474,200 
50,000 
219,400 
50,000 
90,000 


16,244,700 








2009 2008 
Actual Actual 

$ $ 
16,244,700 15,307,600 
8,434,594 8,088,057 
1,844,038 1,910,786 
1,051,024 1,047,624 
1,775,885 1525,747 
298,550 276,514 
332,043 397,196 
2052071 4 201,882 
173,326 159,485 
50,000 50,000 
245,438 228,936 
35,209 ZnO 
88,850 93,513 
14,534,034 14,001,510 
1,710,666 1,306,090 
(1,560,877) (1,607,695) 
(149,789) 301,605 
2,199,729 1,898,124 
2,049,940 2,199,729 





Office of the Auditor General of Ontario 


Statement of Cash Flows 
For the Year Ended March 31, 2009 
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NET INFLOW (OUTFLOW) OF CASH RELATED TO THE 
FOLLOWING ACTIVITIES 


Cash flows from operating activities 
Net excess/(deficiency) of revenue over expenses 


Amortization of capital assets 
Accrued employee benefits obligation 


Changes in non-cash working capital 
Increase in due from Consolidated Revenue Fund 
Increase (decrease) in accounts payable and accrued liabilities 


Investing activities 

Purchase of capital assets 
Net increase (decrease) in cash position 
Cash position, beginning of year 


Cash position, end of year 


/ 


See accompanying notes to financial statements. 





2009 2008 

$ $ 
149,789 (301,605) 
298,550 276,514 
(17,000) 19,000 
431,339 (6,091) 
(289,005) (8,175) 
(89,557) 508,314 
(378,562) 500,039 
(281,339) (309,909) 
(228,562) 184,039 
521,868 337,829 
293,306 521,868 
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Office of the Auditor General of Ontario 


Notes to Financial Statements 
March 31, 2009 





1. Nature of Operations 


In accordance with the provisions of the Auditor General Act and various other statutes and authorities, the 
Auditor General conducts independent audits of government programs, of institutions in the broader public 
sector that receive government grants, and of the fairness of the financial statements of the Province and 
numerous agencies of the Crown. In doing so, the Office of the Auditor General promotes accountability and 
value-for-money in government operations and in broader public sector organizations. 


In addition, under the Government Advertising Act, 2004, the Auditor General is required to review specified types 
of advertising, printed matter or reviewable messages proposed by government offices to determine whether they 
meet the standards required by the Act. 


Under both Acts, the Auditor General reports directly to the Legislative Assembly. 


As required by the Fiscal Transparency and Accountability Act, 2004, the Auditor General was also required to 
review and report on the reasonableness of the 2007 Pre-Election Report prepared by the Ministry of Finance. 
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2. Significant Accounting Policies 


The financial statements have been prepared in accordance with Canadian generally accepted accounting 
principles. The significant accounting policies are as follows: 


(A) ACCRUAL BASIS 


These financial statements are prepared on the accrual basis whereby expenses are recognized in the fiscal year 
that the events giving rise to the expense occur and resources are consumed. 


(B) VOTED APPROPRIATIONS 


The Office is funded through annual voted appropriations from the Province of Ontario. Unspent appropriations 
are returned to the Province’s Consolidated Revenue Fund each year. As the voted appropriation is on a modified 
cash basis, an excess or deficiency of revenue over expenses arises from the application of accrual accounting, 
including the capitalization and amortization of capital assets and the recognition of employee benefit costs 
earned to date that will be funded from future appropriations. 


(C) CAPITAL ASSETS 


Capital assets are recorded at historical cost less accumulated amortization. Amortization of capital assets is 
recorded on the straight-line method over the estimated useful lives of the assets as follows: 


Computer hardware 3 years 
Computer software 3 years 
Furniture and fixtures 5 years 


Leasehold improvements The remaining term of the lease 
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2. Significant Accounting Policies (Continued) 


(D) FINANCIAL INSTRUMENTS 


The Office’s financial instruments consist of cash, due from Consolidated Revenue Fund, accounts payable and 
accrued liabilities, and accrued employee benefits obligation. Under Canadian generally accepted accounting 
principles, financial instruments are classified into one of five categories — available-for-sale, held-for-trading, 
held-to-maturity, loans and receivables, or other financial liabilities. The Office classifies its financial assets and 
liabilities as follows: 


e Cash is classified as held for trading and is recorded at fair value. 


e Due from Consolidated Revenue Fund is classified as loans and receivables and is valued at cost which 
approximates fair value given its short term nature. 


e Accounts payable and accrued liabilities are classified as other financial liabilities and are recorded at cost 
which approximate fair value given their short term maturities. 


e The accrued employee benefits obligation is classified as another financial liability and is recorded at cost 
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based on the entitlements earned by employees up to March 31, 2009. A fair value estimate based on 
actuarial assumptions about when these benefits will actually be paid has not been made as it is not expected 
that there would be a significant difference from the recorded amount. 


It is management’s opinion that the Office is not exposed to any interest rate, currency, liquidity or credit risk 
arising from its financial instruments due to their nature. 


(E) USE OF ESTIMATES 


The preparation of financial statements in accordance with Canadian generally accepted accounting principles 
requires management to make estimates and assumptions that affect the reported amounts of assets and 
liabilities at the date of the financial statements, and the reported amounts of revenue and expenses during the 
reporting period. Actual results could differ from management’s best estimates as additional information 
becomes available in the future. 
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3. Capital Assets 





2009 2008 
Accumulated Net Book Net Book 
Cost Amortization Value Value 
$ $ $ $ 
Computer hardware 586,793 349,057 237,136 202,863 
Computer software 210,058 133,670 76,388 13,837 
Furniture and fixtures 312,846 169,747 143,099 157 cle 
Leasehold improvements 235,868 112,031 123,837 164,457 
1,345,565 764,505 581,060 598,271 


Investment in capital assets represents the accumulated cost of capital assets less accumulated amortization and 
disposals. 


4. Obligation for Future Employee Benefits 
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Although the Office’s employees are not members of the Ontario Public Service, under provisions in the Auditor 
General Act, the Office’s employees are entitled to the same benefits as Ontario Public Service employees. The 
future liability for benefits earned by the Office’s employees is included in the estimated liability for all provincial 
employees that have earned these benefits and is recognized in the Province’s consolidated financial statements. 
These benefits are accounted for as follows: 


(A) PENSION BENEFITS 


The Office’s employees participate in the Public Service Pension Fund (PSPF) which is a defined benefit pension 
plan for employees of the Province and many provincial agencies. The Province of Ontario, which is the sole 
sponsor of the PSPF, determines the Office’s annual payments to the fund. As the sponsor is responsible for 
ensuring that the pension funds are financially viable, any surpluses or unfunded liabilities arising from statutory 
actuarial funding valuations are not assets or obligations of the Office. The Office’s required annual payments of 
$625,585 (2008 - $599,451), are included in employee benefits expense in the Statement of Operations and 
Accumulated Deficit. 
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4. Obligation for Future Employee Benefits (Continued) 
(B) ACCRUED EMPLOYEE BENEFITS OBLIGATION 


Although the costs of any legislated severance and unused vacation entitlements earned by employees are 
recognized by the Province when earned by eligible employees, these costs are also recognized in these financial 
statements. These costs for the year amounted to $108,000 (2008 — $346,000) and are included in employee 
benefits in the Statement of Operations and Accumulated Deficit. The total liability for these costs is reflected in 


the accrued employee benefits obligation, less any amounts payable within one year, which are included in 
accounts payable and accrued liabilities, as follows: 


2009 2008 

$ $ 

Total liability for severance and vacation 2,631,000 2,798,000 
Less: Due within one year and included in 

accounts payable and accrued liabilities (634,000) (784,000) 

Accrued employee benefits obligation 1,997,000 2,014,000 


Chapter 7 


(C) OTHER NON-PENSION POST-EMPLOYMENT BENEFITS 


The cost of other non-pension post-retirement benefits is determined and funded on an ongoing basis by the 
Ontario Ministry of Government Services and accordingly is not included in these financial statements. 


5. Commitment 


The Office has an operating lease to rent premises for an 11-year period, which commenced November 1, 2000. 
The minimum rental commitment for the remaining term of the lease is as follows: 


$ 
2009-10 O2a 00 
2010-11 IZ3;000 
2011-12 306,465 
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6. Public Sector Salary Disclosure Act, 1996 


Section 3(5) of this Act requires disclosure of Ontario public-sector employees paid an annual salary in excess of 
$100,000 in calendar year 2008. 





Taxable 

Salary Benefits 

Name Position $ $ 

McCarter, Jim Auditor General 242,772 384 

Peall, Gary Deputy Auditor General 178,672 295 

Amodeo, Paul Director 136,789 25 

Cheung, Andrew Director 136,789 Zoi 

Chiu, Rudolph Director 120,375 214 

Fitzmaurice, Gerard Director 134,588 231 

Klein, Susan Director 136,350 2a 

' Mazzone, Vince Director 122,549 218 
=| McDowell, John Director 134,588 231 
eB} Mishchenko, Nicholas —_Director 136,789 231 
& | Sciarra, John Director of Operations 120,315 214 
Bell, Laura Audit Manager 105,467 186 
Brennan, Michael Audit Manager 108,573 186 

Chagani, Hassnain Audit Manager 108,573 186 

Cumbo, Wendy Audit Manager 108,573 186 

Gotsis, loanna Audit Manager 108,573 186 

MacNeil, Richard Audit Manager 107,509 186 

Mok, Rita Audit Manager 108,573 186 

Pelow, William Audit Manager 104,607 182 

Rogers, Fraser Audit Manager 110,349 186 

Tersigni, Anthony Audit Manager 108,573 186 

Young, Denise Audit Manager 108,573 186 

Wiebe, Annemarie Manager, Human Resources 108,573 186 


7. Reconciliation to Public Accounts Volume 1 Basis of Presentation 


The Office’s Statement of Expenses presented in Volume 1 of the Public Accounts of Ontario was prepared on a 
basis consistent with the accounting policies followed for the Province’s financial statements, under which 
purchases of computers and software and of leasehold improvements are expensed in the year of acquisition 
rather than being capitalized and amortized over their useful lives. Volume 1 also excludes the accrued employee 
future benefit costs recognized in these financial statements as well as in the Province’s summary financial 
statements. A reconciliation of total expenses reported in volume 1 to the total expenses reported in these 
financial statements is as follows: 
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7. Reconciliation to Public Accounts Volume 1 Basis of Presentation (Continued) 


2009 2008 

$ $ 

Total expenses per Public Accounts Volume 1 14,683,823 13,699,905 
purchase of capital assets (281,339) (309,909) 
amortization of capital assets 298,550 276,514 
change in accrued future employee benefit costs (167,000) 335,000 

Total expenses per audited financial statements 14,534,034 14,001,510 


8. Management of Capital 


The Office’s capital consists of cash. In managing cash the Office maintains sufficient funds to meet estimated 
cash requirements each month and requisitions the necessary amount from the Ministry of Finance on a monthly 
basis. The Office’s bank account is pooled with other government accounts for cash management purposes in 
order to reduce the province’s borrowing requirements and/or to earn interest. Accordingly, the Office’s capital 
is not at risk. 





| 
he 
oD 
_ 
, 2 
, cs 
= 
oO 


9. Comparative Figures 


Certain of the 2007/08 figures have been reclassified to conform to the presentation adopted for the 2008/09 
fiscal year. 
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Exhibit 1 


1. Agencies whose accounts are audited 
by the Auditor General 


AgriCorp 

Algonquin Forestry Authority 

Cancer Care Ontario 

Centennial Centre of Science and Technology 

Chief Electoral Officer, Election Finances Act 

Election Fees and Expenses, Election Act 

Financial Services Commission of Ontario 

Grain Financial Protection Board, Funds for 
Producers of Grain Corn, Soybeans, Wheat, and 
Canola . 

Investor Education Fund, Ontario Securities 
Commission 

Legal Aid Ontario 

Liquor Control Board of Ontario 

Livestock Financial Protection Board, Fund for 
Livestock Producers 

Northern Ontario Heritage Fund Corporation 

North Pickering Development Corporation 

Office of the Assembly 

Office of the Children’s Lawyer 

Office of the Environmental Commissioner 

Office of the Information and Privacy 
Commissioner 

Office of the Ombudsman 

Ontario Clean Water Agency (December 31)* 





* Dates in parentheses indicate fiscal periods ending on a 
date other than March 31. 


Agencies of the Crown 


Ontario Development Corporation 

Ontario Educational Communications Authority 

Ontario Electricity Financial Corporation 

Ontario Energy Board 

Ontario Financing Authority 

Ontario Food Terminal Board 

Ontario Heritage Trust 

Ontario Immigrant Investor Corporation 

Ontario Media Development Corporation 

Ontario Mortgage and Housing Corporation 

Ontario Northland Transportation Commission 

Ontario Place Corporation 

Ontario Racing Commission 

Ontario Realty Corporation 

Ontario Securities Commission 

Owen Sound Transportation Company Limited 

Pension Benefits Guarantee Fund, Financial 
Services Commission of Ontario 

Province of Ontario Council for the Arts 

Provincial Advocate for Children and Youth 

Provincial Judges Pension Fund, Provincial Judges 
Pension Board 

Public Guardian and Trustee for the Province of 
Ontario 

Toronto Area Transit Operating Authority 


2. Agencies whose accounts are audited 
by another auditor under the direction of 
the Auditor General 


Motor Vehicle Accident Claims Fund 

Niagara Parks Commission (October 31)* 

Ontario Mental Health Foundation 

St. Lawrence Parks Commission 

Workplace Safety and Insurance Board 
(December 31)* 


* Dates in parentheses indicate fiscal periods ending on a 
date other than March 31. 
Note: 
The following changes were made during the 2008/09 
fiscal year: 
Deletion: 
Chief Electoral Officer, Electoral System Referendum Act 
Ontario Mortgage Corporation 


TVOntario Foundation 
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Corporations whose accounts are 
audited by an auditor other than the 
Auditor General, with full access by the 
Auditor General to audit reports, working 
papers, and other related documents 


Agricultural Research Institute of Ontario 

Art Gallery of Ontario Crown Foundation 

Board of Funeral Services 

Brock University Foundation 

Canadian Opera Company Crown Foundation 

Canadian Stage Company Crown Foundation 

Central Community Care Access Centre 

Central East Community Care Access Centre 

Central East Local Health Integration Network 

Central Local Health Integration Network 

Central West Community Care Access Centre 

Central West Local Health Integration Network 

Champlain Community Care Access Centre 

Champlain Local Health Integration Network 

Deposit Insurance Corporation of Ontario 

eHealth Ontario 

Echo: Improving Women’s Health in Ontario 

Education Quality and Accountability Office 

Erie St. Clair Community Care Access Centre _ 

Erie St. Clair Local Health Integration Network 

Foundation at Queen’s University at Kingston 

Greater Toronto Transit Authority 

Hamilton Niagara Haldimand Brant Community 
Care Access Centre 

Hamilton Niagara Haldimand Brant Local Health 
Integration Network 

HealthforceOntario Marketing and Recruitment 
Agency 


Crown-controlled 
Corporations 


Higher Education Quality Council of Ontario 

Hydro One Inc. 

Independent Electricity System Operator 

McMaster University Foundation 

McMichael Canadian Art Collection 

Metrolinx 

Metropolitan Toronto Convention Centre 
Corporation 

Mississauga Halton Community Care Access Centre 

Mississauga Halton Local Health Integration 
Network 

Municipal Property Assessment Corporation 

National Ballet of Canada Crown Foundation 

North East Community Care Access Centre 

North East Local Health Integration Network 

North Simcoe Muskoka Community Care Access 
Centre 

North Simcoe Muskoka Local Health Integration 
Network 

North West Community Care Access Centre 

North West Local Health Integration Network 

Northern Ontario Grow Bonds Corporation 

Ontario Agency for Health Protection and 
Promotion 

Ontario Capital Growth Corporation 

Ontario Foundation for the Arts 

Ontario French Language Communications 
Authority 

Ontario Health Quality Council 

Ontario Infrastructure Projects Corporation 

Ontario Lottery and Gaming Corporation 

Ontario Mortgage Corporation 

Ontario Pension Board 


Ontario Power Authority 

Ontario Power Generation Inc. 

Ontario Tourism Marketing Partnership 
Corporation 

Ontario Trillium Foundation 

Ottawa Congress Centre 

Royal Botanical Gardens Crown Foundation 

Royal Ontario Museum 

Royal Ontario Museum Crown Foundation 

Science North 

Shaw Festival Crown Foundation 

Stadium Corporation of Ontario Limited 

South East Community Care Access Centre 

South East Local Health Integration Network 

South West Community Care Access Centre 





Note: 
The following changes were made during the 2008/09 
fiscal year: 
Addition: 
eHealth Ontario 
Metrolinx 
Ontario Capital Growth Corporation 
Ontario French Language Communications Authority 
Ontario Mortgage Corporation 
Deletion: 
Baycrest Hospital Crown Foundation 
Mount Sinai Hospital Crown Foundation 
North York General Hospital Crown Foundation 
St. Clair Parks Commission 
Smart Systems for Health Agency 
Sunnybrook Hospital Crown Foundation 
Toronto East General Hospital Crown Foundation 
Toronto Hospital Crown Foundation 
Women’s College and Wellesley Central Crown 


Foundation 
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South West Local Health Integration Network 

Stratford Festival Crown Foundation 

Toronto Central Community Care Access Centre 

Toronto Central Local Health Integration Network 

Toronto Islands Residential Community Trust 
Corporation 

Toronto Symphony Orchestra Crown Foundation 

Toronto Waterfront Revitalization Corporation 

Trent University Foundation 

Trillium Gift of Life Network 

University of Ottawa Foundation 

Walkerton Clean Water Centre 

Waterfront Regeneration Trust Agency 

Waterloo Wellington Community Care Access Centre 

Waterloo Wellington Local Health Integration 
Network 









Exhibit 3 
_ Treasury Board Orders 


Under subsection 12(2)(e) of the Auditor General in the following table. While ministries may track 
Act, the Auditor General is required to annually expenditures related to these orders in more detail 
report all orders of the Treasury Board made to by creating accounts at the sub-vote and item level, 
authorize payments in excess of appropriations, this schedule summarizes such expenditures at the 
stating the date of each order, the amount author- vote and item level. 


ized, and the amount expended. These are outlined 


Ministry DateofOrder Authorized ($) Expended ($) 


Aboriginal Affairs Nov. 20, 2008 4,900,000 = 
Apr. 9, 2009 1,200,000 1,199,000 

6,100,000 —‘1,199,000 

Agriculture, Food and Rural Affairs Feb. 5, 2009 37,800,000 32,668,967 
Mar. 30, 2009 1,700,000 317,782 

Apr. 16, 2009 110,000 - 


39,610,000 _—=—- 32,986,749 






Attorney General Mar. 12, 2009 77,856, 700 65,582,167 
Mar. 12, 2009 8,633,300 4,782,478 

— 86,490,000 70,364,645 

Cabinet Office May 8, 2008 100,000 100,000 
May 15, 2008 1,000,000 23,839 






1,100,000 = 123,839 






Children and Youth Services Jun. 11, 2008 115535300 _ 
Jun. 24, 2008 400,000 = 

Oct. 2, 2008 756,600 133,759 

Dec. 4, 2008 6,748,300 6,748,300 

Feb. 26, 2009 59,811,400 59,811,400 

Feb. 26, 2009 14,791,700 13,806,300 

Mar. 12, 2009 7,000,000 3,899,436 

Mar. 19, 2009 644,900 . - 

Apr. 9, 2009 4,105,600 607,369 

105,811,800 85,006,564 





Citizenship and Immigration Apr. 9, 2009 «2,255,100 2,093,091 
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Ministry 
Community and Social Services 


Community Safety and Correctional Services 


Culture 


Economic Development 


Education 


Energy and Infrastructure 


Environment 


Finance 


Date of Order 
Jan. 29, 2009 
Jan. 29, 2009 
Apr. 9, 2009 
Apr. 9, 2009 


Sep. 25, 2008 
Nov. 20, 2008 
Dec. 4, 2008 
Mar. 12, 2009 
Apr. 6, 2009 
Apr. 20, 2009 


Aug. 19, 2008 
Apr. 9, 2009 


Mar. 12, 2009 
Apr. 16, 2009 


Sep. 25, 2008 
Feb. 23, 2009 
Feb. 23, 2009 


Dec. 4, 2008 
Dec. 4, 2008 
Mar. 12, 2009 
Apr. 16, 2009 


Dec. 4, 2008 
Mar. 12, 2009 


Aug. 19, 2008 
Aug. 19, 2008 
Aug. 19, 2008 
Dec. 4, 2008 
Dec. 4, 2008 
Jan. 29, 2009 
Jan. 30, 2009 
Feb. 23, 2009 
Mar. 12, 2009 
Mar. 18, 2009 
Apr. 16, 2009 


Authorized ($) 
58,643,700 
241,910,000 
13,754,800 
2,900,000 
317,208,500 


31,000,000 
3,500,000 
440,000 
66,114,100 
7,150,000 
1,350,000 
109,554,100 


1,710,000 
1,141,800 
2,851,800 


305,000 
3,052,900 
3,357,900 


337,600 
1,758,700 
281,500,000 
283,596,300 


19,500,000 
82,493,600 
130,221,600 
228,400 


_ 232,443,600 


39,000,000 
11,851,300 
50,851,300 


495,300 
1,310,300 
194,800 
100,280,300 
900,000 
537,861,600 
500,000 
333,900 
189,105,300 
3,173,000 
592,000 
834,746,500 
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Expended ($) 
54,322,821 
241,777,257 

7,875,800 


303,975,878 


13,788,137 
3,500,000 
440,000 
51,551,345 
5,000,634 


74,280,116 


884,760 
884,760 


1,758,700 
160,220,238 
161,978,938 


14,745,807 


14,745,807 


39,000,000 
11,720,414 
50,720,414 


900,000 


500,000 


3,173,000 


4,573,000 
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‘Ministry Date of Order Authorized ($) Expended ($) 
Government Services Jul. 31, 2008 998,600 387,345 
Aug. 19, 2008 4,065,800 28,240 
Aug. 19, 2008 1,612,500 1,612,500 
Aug. 19, 2008 6,193,400 6,193,400 
Aug. 19, 2008 3,058,000 3,058,000 
Aug. 19, 2008 506,000 — 
Aug. 31, 2008 1,000,000 1,000,000 
Mar. 19, 2009 95,483,600 93,325,834 
Apr. 1, 2009 17,663,100 16,623,850 
Apr. 16, 2009 2,427,500 1,159,411 


Sep. 17, 2009 126,473,400 126,472,401 





259,481,900 249,860,981 

Health and Long—Term Care Nov. 21, 2008 2,664,500 2,606,993 
Feb. 11, 2009 10,904,600 9,023,038 

Feb. 26, 2009 2,370,700 - 

Feb. 26, 2009 694,287,000 654,898,839 

Mar. 12, 2009 558,776,700 558,776,600 

Mar. 19, 2009 937,300 _ 

Sep. 17, 2009 338,382,400 328,195,236 








1,608,323,100 _1,553,500,706 

Health Promotion Aug. 27, 2008 900,000 - 
Oct. 2, 2008 4,100,000 - 

Nov. 20, 2008 3,900,000 ae: 
2 8900000 











Labour Feb. 10, 2009 2,466,200 —=_1,506,983 
Municipal Affairs and Housing Jun. 24, 2008 2,000,000 2,000,000 
Jul. 23, 2008 1,784,000 1,784,000 
Aug. 19, 2008 5,003,300 - 
Nov. 27, 2008 2,412,000 2,412,000 
Dec. 4, 2008 1,000,000 ~ 
Mar. 24, 2009 5,580,200 4,536,720 
Mar. 26, 2009 1,240,000 391,120 





"49,019,500 11,123,840 








Natural Resources May 29, 2008 Se BA00 000 

Northern Development and Mines Jun. 24, 2008 11,000,000 8,897,895 
Mar. 12, 2009 700,000 - 
Mar. 19, 2009 1,150,000 240,309 





12,850,000 (9,138,204 





Office of Francophone Affairs Aug. 19, 2008 52,500 49,530 
Sep. 25, 2008 51,000 - 
Dec. 4, 2008 150,000 


co «253,500 49,530 





UTES ta Date of Order 


Revenue Aug. 19, 2008 
Small Business and Consumer Services Sep. 25, 2008 
Mar. 12, 2009 


Tourism Jul. 23, 2008 
Jan. 29, 2009 
Feb. 26, 2009 
Apr. 9, 2009 
Apr. 16, 2009 


Training, Colleges and Universities Aug. 19, 2008 
Aug. 19, 2008 
Aug. 19, 2008 
Aug. 19, 2008 
Feb. 23, 2009 
Mar. 12, 2009 
Mar. 26, 2009 
Apr. 8, 2009 

Apr. 16, 2009 


Transportation Feb. 23, 2009 
Feb. 26, 2009 


Mar. 25, 2009 


4,260,340,900 
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Authorized (S$) Expended ($) 
750,000 - 
2,000,000 2,000,000 
1,000,000 376,576 
3,000,000 _ 2,376,576 
1,310,000 1,310,000 
5,336,700 = 
862,600 828,547 
57,600 es 
2,823,800 = 
10,390,700 2,138,547 
1,059,900 2 
380,800 380,800 
4,800,000 a 
5,884,900 5,580,900 
4,647,900 = 
1,110,000 1,075,407 
120,000,000 = 
2,455,400 1,680,044 
1,735,200 = 
142,074,100 = 8,717,151 
11,625,000 11,625,000 
88,800,000 45,651,708 
11,330,000 9,240,671 
111,755,000 66,517,379 





Total Treasury Board Orders 


2,707,862,698 
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